[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:20 dist-test-slave-gjw7 krb5kdc[1867](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:20 dist-test-slave-gjw7 krb5kdc[1867](info): set up 2 sockets
Aug 08 00:37:20 dist-test-slave-gjw7 krb5kdc[1867](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 00:37:21.395665 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:21 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:21.525274 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46637
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:46637
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:21.625237 1889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:21.625486 1889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:21.625509 1889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:21.626672 1889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:21.626718 1889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:21.626731 1889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:21.626742 1889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:21.626752 1889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:21.628459 1889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:46637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:21.628813 1889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:21.629082 1889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:37:21.636561 1889 server_base.cc:1034] running on GCE node
W20260808 00:37:21.637418 1900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:21.637753 1905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:21.637796 1898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:21.638175 1889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:21.638598 1889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:21.639740 1889 hybrid_clock.cc:648] HybridClock initialized: now 1786149441639731 us; error 31 us; skew 500 ppm
Aug 08 00:37:21 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:21.642594 1889 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:21.643267 1889 webserver.cc:460] Webserver started at http://127.1.208.62:43187/ using document root <none> and password file <none>
I20260808 00:37:21.643555 1889 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:21.643611 1889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:21.643744 1889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:21.645022 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "34d34ea54215441fb4412e8b985575bc"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "6167018d6be8bfd8f51f2062dd5cd434"
server_key_iv: "b93b878267437fca0bd169691c2bff46"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.645499 1889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "34d34ea54215441fb4412e8b985575bc"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "6167018d6be8bfd8f51f2062dd5cd434"
server_key_iv: "b93b878267437fca0bd169691c2bff46"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.647190 1889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:37:21.647913 1919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:21.648078 1889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:21.648168 1889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "34d34ea54215441fb4412e8b985575bc"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "6167018d6be8bfd8f51f2062dd5cd434"
server_key_iv: "b93b878267437fca0bd169691c2bff46"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.648236 1889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:21.658545 1889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:21.660636 1889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:21.660800 1889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:21.668601 1889 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:46637
I20260808 00:37:21.668663 2006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:46637 every 8 connection(s)
I20260808 00:37:21.669098 1889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:21.670311 2008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:21.671778 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1889
I20260808 00:37:21.671902 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:21.672735 1856 external_mini_cluster.cc:1428] Setting key 4b4d2ba741c295f2df350a48f776fe1e
I20260808 00:37:21.673848 2008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc: Bootstrap starting.
I20260808 00:37:21.674726 2008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:21.675094 2008 log.cc:824] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:21.675890 2008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc: No bootstrap required, opened a new log
I20260808 00:37:21.678092 2008 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } }
I20260808 00:37:21.678253 2008 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:21.678293 2008 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34d34ea54215441fb4412e8b985575bc, State: Initialized, Role: FOLLOWER
I20260808 00:37:21.678435 2008 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } }
I20260808 00:37:21.678524 2008 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:21.678555 2008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:21.678604 2008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:21.679780 2008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } }
I20260808 00:37:21.679939 2008 leader_election.cc:302] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34d34ea54215441fb4412e8b985575bc; no voters:
I20260808 00:37:21.680155 2008 leader_election.cc:288] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:21.680194 2016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:21.680383 2016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [term 1 LEADER]: Becoming Leader. State: Replica: 34d34ea54215441fb4412e8b985575bc, State: Running, Role: LEADER
I20260808 00:37:21.680532 2016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } }
I20260808 00:37:21.680536 2008 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:21.681116 2018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34d34ea54215441fb4412e8b985575bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } } }
I20260808 00:37:21.681205 2018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:21.681764 2019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34d34ea54215441fb4412e8b985575bc. Latest consensus state: current_term: 1 leader_uuid: "34d34ea54215441fb4412e8b985575bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34d34ea54215441fb4412e8b985575bc" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46637 } } }
I20260808 00:37:21.681838 2019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:21.682039 2026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:21.682789 2026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:21.686779 2026 catalog_manager.cc:1347] Generated new cluster ID: bf139118e1b14c02a369c30a2aae26d7
I20260808 00:37:21.686843 2026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:21 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:21.691358 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:21.673946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35548 (local address 127.1.208.62:46637)
0808 00:37:21.674233 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:21.674243 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:21.674397 (+ 154us) server_negotiation.cc:407] Connection header received
0808 00:37:21.677206 (+ 2809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:21.677216 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:21.677284 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:21.677614 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:21.679342 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.681154 (+ 1812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.683003 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.683520 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.683634 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:21.687871 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:21.687875 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:21.687875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:21.687887 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:21.689548 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:21.689970 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:21.689973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:21.689973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:21.690065 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:21.690302 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:21.690303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:21.690304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:21.690547 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:21.690713 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:21.690995 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 00:37:21.691130 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":160,"threads_started":1}
I20260808 00:37:21.699285 2026 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:21.699826 2026 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:21.707087 2026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34d34ea54215441fb4412e8b985575bc: Generated new TSK 0
I20260808 00:37:21.707298 2026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:21.744756 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:21.860044 2054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:21.860268 2054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:21.860302 2054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:21.861550 2054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:21.861595 2054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:21.861634 2054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:21.863176 2054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:21.863468 2054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:21.863703 2054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:21.868744 2063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:21.868739 2062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:21.868991 2054 server_base.cc:1034] running on GCE node
W20260808 00:37:21.868779 2066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:21.869258 2054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:21.869568 2054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:21.870728 2054 hybrid_clock.cc:648] HybridClock initialized: now 1786149441870709 us; error 30 us; skew 500 ppm
Aug 08 00:37:21 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:21.873222 2054 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:21.873752 2054 webserver.cc:460] Webserver started at http://127.1.208.1:36141/ using document root <none> and password file <none>
I20260808 00:37:21.873987 2054 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:21.874037 2054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:21.874171 2054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:21.875409 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "eff4caf8e30e4fec94d46a11b862d51f"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "ab1cab6bfb7a5ba638b60524e283d05d"
server_key_iv: "db3d5c2a8ae28fbbfdc3799be5cc3201"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.875809 2054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "eff4caf8e30e4fec94d46a11b862d51f"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "ab1cab6bfb7a5ba638b60524e283d05d"
server_key_iv: "db3d5c2a8ae28fbbfdc3799be5cc3201"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.877218 2054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:21.877978 2081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:21.878125 2054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:21.878214 2054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "eff4caf8e30e4fec94d46a11b862d51f"
format_stamp: "Formatted at 2026-08-08 00:37:21 on dist-test-slave-gjw7"
server_key: "ab1cab6bfb7a5ba638b60524e283d05d"
server_key_iv: "db3d5c2a8ae28fbbfdc3799be5cc3201"
server_key_version: "encryptionkey@0"
I20260808 00:37:21.878276 2054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:21.891911 2054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:21.894158 2054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:21.894315 2054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:21.894644 2054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:21.895056 2054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:21.895100 2054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:21.895133 2054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:21.895159 2054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:21.911437 2054 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35273
I20260808 00:37:21.911615 2266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35273 every 8 connection(s)
I20260808 00:37:21.911934 2054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:21.919129 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2054
I20260808 00:37:21.919214 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:21.919355 1856 external_mini_cluster.cc:1428] Setting key 81368141d150718c129c2f0ec8a9fa77
Aug 08 00:37:21 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:21.923002 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:21.912904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60765 (local address 127.1.208.62:46637)
0808 00:37:21.912966 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:21.912968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:21.913504 (+ 536us) server_negotiation.cc:407] Connection header received
0808 00:37:21.914457 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:21.914459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:21.914472 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:21.914533 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:21.915804 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.916321 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.917401 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.917628 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.917684 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:21.920221 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:21.920222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:21.920223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:21.920231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:21.921573 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:21.922065 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:21.922066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:21.922067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:21.922123 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:21.922366 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:21.922366 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:21.922367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:21.922486 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:21.922542 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:21.922810 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 00:37:21.922910 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:37:21.923197 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:21.912847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46637 (local address 127.1.208.1:60765)
0808 00:37:21.913259 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:21.913378 (+ 119us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:21.914341 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:21.914613 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:21.914617 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:21.915449 (+ 832us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:21.915693 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:21.915699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.916415 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.916416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:21.917277 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:21.917279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:21.917701 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:21.917702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:21.917844 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:21.918391 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:21.918401 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:21.920125 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:21.921653 (+ 1528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:21.921663 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:21.921666 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:21.921995 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:21.922185 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:21.922186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:21.922186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:21.922306 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:21.922541 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:21.922549 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:21.922691 (+ 142us) client_negotiation.cc:769] Sending connection context
0808 00:37:21.922783 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:37:21.922951 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
I20260808 00:37:21.923955 2269 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46637
I20260808 00:37:21.924054 2269 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:21.924240 2269 heartbeater.cc:511] Master 127.1.208.62:46637 requested a full tablet report, sending...
I20260808 00:37:21.924916 1943 ts_manager.cc:194] Registered new tserver with Master: eff4caf8e30e4fec94d46a11b862d51f (127.1.208.1:35273)
I20260808 00:37:21.925635 1943 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:60765
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:21.949738 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:22.043788 2275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:22.044008 2275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:22.044034 2275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:22.045148 2275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:22.045183 2275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:22.045209 2275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:22.046759 2275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:22.046967 2275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:22.047166 2275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:22.052549 2288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:22.052503 2287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:22.052524 2293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:22.052826 2275 server_base.cc:1034] running on GCE node
I20260808 00:37:22.053043 2275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:22.053320 2275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:22.054497 2275 hybrid_clock.cc:648] HybridClock initialized: now 1786149442054479 us; error 29 us; skew 500 ppm
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:22.057078 2275 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:22.057765 2275 webserver.cc:460] Webserver started at http://127.1.208.2:42147/ using document root <none> and password file <none>
I20260808 00:37:22.057981 2275 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:22.058027 2275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:22.058130 2275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:22.059154 2275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "c17fd1a5d53342eabddef4cf0c8ad924"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "d6a7559c8d8048a2d6cf3cc59777fd83"
server_key_iv: "193113f634694d2f10304e6be1b43c10"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.059469 2275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "c17fd1a5d53342eabddef4cf0c8ad924"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "d6a7559c8d8048a2d6cf3cc59777fd83"
server_key_iv: "193113f634694d2f10304e6be1b43c10"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.060834 2275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:22.061597 2306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.061746 2275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:22.061836 2275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "c17fd1a5d53342eabddef4cf0c8ad924"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "d6a7559c8d8048a2d6cf3cc59777fd83"
server_key_iv: "193113f634694d2f10304e6be1b43c10"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.061895 2275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:22.091671 2275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:22.093966 2275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:22.094094 2275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:22.094372 2275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:22.094751 2275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:22.094789 2275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.094812 2275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:22.094823 2275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.106521 2275 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:40111
I20260808 00:37:22.106604 2496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:40111 every 8 connection(s)
I20260808 00:37:22.106933 2275 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:22.116317 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.107728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:42081 (local address 127.1.208.62:46637)
0808 00:37:22.107786 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:22.107788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:22.108003 (+ 215us) server_negotiation.cc:407] Connection header received
0808 00:37:22.108527 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:22.108528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:22.108538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:22.108590 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:22.109602 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.110106 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.111079 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.111266 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.111313 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.113546 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:22.113547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:22.113548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:22.113555 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:22.114864 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.115273 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.115274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.115274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.115336 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.115565 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.115566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.115567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.115700 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:22.115763 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:22.116113 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 00:37:22.116209 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:37:22.116460 2502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.107699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46637 (local address 127.1.208.2:42081)
0808 00:37:22.107868 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:22.107920 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:22.108443 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:22.108649 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:22.108653 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:22.109241 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:22.109530 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:22.109537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.110170 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.110171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:22.111010 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:22.111013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.111323 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.111324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:22.111427 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.111916 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:22.111924 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:22.113440 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:22.114943 (+ 1503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:22.114952 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:22.114954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:22.115181 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:22.115396 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:22.115396 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:22.115397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:22.115509 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:22.115784 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:22.115791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:22.115966 (+ 175us) client_negotiation.cc:769] Sending connection context
0808 00:37:22.116070 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 00:37:22.116234 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260808 00:37:22.116876 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2275
I20260808 00:37:22.116915 2497 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46637
I20260808 00:37:22.116979 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:22.117007 2497 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:22.117151 1856 external_mini_cluster.cc:1428] Setting key fc8d7fb6a7aa6288fce516efbd5dd7a9
I20260808 00:37:22.117194 2497 heartbeater.cc:511] Master 127.1.208.62:46637 requested a full tablet report, sending...
I20260808 00:37:22.117638 1942 ts_manager.cc:194] Registered new tserver with Master: c17fd1a5d53342eabddef4cf0c8ad924 (127.1.208.2:40111)
I20260808 00:37:22.118100 1942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:42081
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:22.146389 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:22.249432 2506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:22.249648 2506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:22.249683 2506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:22.250965 2506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:22.251009 2506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:22.251049 2506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:22.252822 2506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46637
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:22.253080 2506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:22.253320 2506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:22.259025 2515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:22.259099 2513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:22.259027 2520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:22.259065 2506 server_base.cc:1034] running on GCE node
I20260808 00:37:22.259441 2506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:22.259682 2506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:22.260824 2506 hybrid_clock.cc:648] HybridClock initialized: now 1786149442260812 us; error 30 us; skew 500 ppm
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:22.263661 2506 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:22.264282 2506 webserver.cc:460] Webserver started at http://127.1.208.3:45395/ using document root <none> and password file <none>
I20260808 00:37:22.264679 2506 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:22.264758 2506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:22.264927 2506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:22.266451 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "c714314349904be5b0ceb0cd876f16d8"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "9fd26545f5f4b987c8432bc3fea974b8"
server_key_iv: "c5d99d65e17f52c8bcf263f17c374e3c"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.266909 2506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "c714314349904be5b0ceb0cd876f16d8"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "9fd26545f5f4b987c8432bc3fea974b8"
server_key_iv: "c5d99d65e17f52c8bcf263f17c374e3c"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.268360 2506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:37:22.269171 2528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.269356 2506 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:22.269446 2506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c714314349904be5b0ceb0cd876f16d8"
format_stamp: "Formatted at 2026-08-08 00:37:22 on dist-test-slave-gjw7"
server_key: "9fd26545f5f4b987c8432bc3fea974b8"
server_key_iv: "c5d99d65e17f52c8bcf263f17c374e3c"
server_key_version: "encryptionkey@0"
I20260808 00:37:22.269511 2506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:22.280728 2506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:22.283123 2506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:22.283327 2506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:22.283740 2506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:22.284113 2506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:22.284152 2506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.284180 2506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:22.284197 2506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:22.303612 2506 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:41967
I20260808 00:37:22.303764 2733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:41967 every 8 connection(s)
I20260808 00:37:22.304111 2506 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:22.314059 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2506
I20260808 00:37:22.314147 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:22.314312 1856 external_mini_cluster.cc:1428] Setting key b5f84f6fdfde93ade26901e9d4835e92
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:22.318733 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.305064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:60619 (local address 127.1.208.62:46637)
0808 00:37:22.305153 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:22.305155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:22.305566 (+ 411us) server_negotiation.cc:407] Connection header received
0808 00:37:22.306449 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:22.306451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:22.306464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:22.306523 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:22.307810 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.308327 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.309601 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.309812 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.309878 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.315694 (+ 5816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:22.315696 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:22.315697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:22.315706 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:22.317358 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.317766 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.317767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.317767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.317821 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.318147 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.318149 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.318149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.318285 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:22.318346 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:22.318590 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 00:37:22.318661 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:37:22.318992 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.305035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46637 (local address 127.1.208.3:60619)
0808 00:37:22.305389 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:22.305434 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:22.306313 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:22.306597 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:22.306601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:22.307355 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:22.307701 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:22.307709 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.308399 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.308400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:22.309503 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:22.309506 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.309901 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.309902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:22.310016 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.313712 (+ 3696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:22.313723 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:22.315587 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:22.317451 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:22.317459 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:22.317461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:22.317697 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:22.317900 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:22.317901 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:22.317901 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:22.318042 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:22.318341 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:22.318345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:22.318480 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 00:37:22.318560 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:37:22.318705 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":179,"threads_started":1}
I20260808 00:37:22.319577 2735 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46637
I20260808 00:37:22.319700 2735 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:22.319916 2735 heartbeater.cc:511] Master 127.1.208.62:46637 requested a full tablet report, sending...
I20260808 00:37:22.320294 1942 ts_manager.cc:194] Registered new tserver with Master: c714314349904be5b0ceb0cd876f16d8 (127.1.208.3:41967)
I20260808 00:37:22.320680 1942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:60619
I20260808 00:37:22.326589 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:22.356010 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.347490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50996 (local address 127.1.208.1:35273)
0808 00:37:22.347749 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:22.347755 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:22.347793 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:37:22.347864 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:22.347866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:22.347881 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:22.348013 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:22.348530 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.349524 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.350214 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.350644 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.350701 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.352878 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:22.352887 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:22.352888 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:22.352894 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:22.354813 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.355110 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.355113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.355114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.355168 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.355384 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.355384 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.355385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.355525 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:22.355755 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:22.355796 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:37:22.355863 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
W20260808 00:37:22.356453 2120 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50996
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[1867](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:22.382822 2750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:22.375319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51006 (local address 127.1.208.1:35273)
0808 00:37:22.375372 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:22.375374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:22.375437 (+ 63us) server_negotiation.cc:407] Connection header received
0808 00:37:22.375500 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:22.375501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:22.375514 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:22.375572 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:22.376233 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.376757 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.377533 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:22.377744 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:22.377808 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:22.380208 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:22.380210 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:22.380210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:22.380218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:22.381822 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.382163 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.382165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.382165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.382226 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:22.382397 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:22.382398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:22.382398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:22.382494 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:22.382603 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:22.382673 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:37:22.382752 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:37:22.383373 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2054
I20260808 00:37:22.405001 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2275
I20260808 00:37:22.417917 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2506
I20260808 00:37:22.427673 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1889
2026-08-08T00:37:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2324 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[2763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[2763](info): set up 2 sockets
Aug 08 00:37:22 dist-test-slave-gjw7 krb5kdc[2763](info): commencing operation
krb5kdc: starting...
W20260808 00:37:23.617517 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:23 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:23.742497 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37291
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37291
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:23.826150 2792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:23.826375 2792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:23.826396 2792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:23.827522 2792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:23.827557 2792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:23.827570 2792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:23.827581 2792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:23.827592 2792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:23.829087 2792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:23.829329 2792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:23.829528 2792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:23.834498 2798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:23.834478 2801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:23.834474 2806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:23.834851 2792 server_base.cc:1034] running on GCE node
I20260808 00:37:23.835101 2792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:23.835387 2792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:23.836550 2792 hybrid_clock.cc:648] HybridClock initialized: now 1786149443836530 us; error 30 us; skew 500 ppm
Aug 08 00:37:23 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:23.839252 2792 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:23.839702 2792 webserver.cc:460] Webserver started at http://127.1.208.62:37179/ using document root <none> and password file <none>
I20260808 00:37:23.839937 2792 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:23.839983 2792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:23.840116 2792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:23.841368 2792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "14e67237fdcb495ea3ce75cab580de76"
format_stamp: "Formatted at 2026-08-08 00:37:23 on dist-test-slave-gjw7"
server_key: "330cb45ad50a6cacf6a75e251afb9999"
server_key_iv: "f2132e47fcfedcefc38c3497a682751a"
server_key_version: "encryptionkey@0"
I20260808 00:37:23.841727 2792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "14e67237fdcb495ea3ce75cab580de76"
format_stamp: "Formatted at 2026-08-08 00:37:23 on dist-test-slave-gjw7"
server_key: "330cb45ad50a6cacf6a75e251afb9999"
server_key_iv: "f2132e47fcfedcefc38c3497a682751a"
server_key_version: "encryptionkey@0"
I20260808 00:37:23.843055 2792 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:23.843837 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:23.844022 2792 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:23.844111 2792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "14e67237fdcb495ea3ce75cab580de76"
format_stamp: "Formatted at 2026-08-08 00:37:23 on dist-test-slave-gjw7"
server_key: "330cb45ad50a6cacf6a75e251afb9999"
server_key_iv: "f2132e47fcfedcefc38c3497a682751a"
server_key_version: "encryptionkey@0"
I20260808 00:37:23.844170 2792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:23.868955 2792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:23.870996 2792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:23.871119 2792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:23.875129 2792 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37291
I20260808 00:37:23.875166 2907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37291 every 8 connection(s)
I20260808 00:37:23.875524 2792 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:23.876438 2908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:23.878242 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2792
I20260808 00:37:23.878338 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:23.878502 1856 external_mini_cluster.cc:1428] Setting key 19269e70ff204686dc8d740f30d1b3b3
I20260808 00:37:23.879160 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76: Bootstrap starting.
I20260808 00:37:23.879920 2908 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:23.880249 2908 log.cc:824] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:23.881208 2908 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76: No bootstrap required, opened a new log
I20260808 00:37:23.883989 2908 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } }
I20260808 00:37:23.884182 2908 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:23.884249 2908 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e67237fdcb495ea3ce75cab580de76, State: Initialized, Role: FOLLOWER
I20260808 00:37:23.884431 2908 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } }
I20260808 00:37:23.884555 2908 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:23.884608 2908 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:23.884678 2908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:23.885411 2908 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } }
I20260808 00:37:23.885560 2908 leader_election.cc:302] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14e67237fdcb495ea3ce75cab580de76; no voters:
I20260808 00:37:23.885773 2908 leader_election.cc:288] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:23.885828 2921 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:23.886049 2908 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:23.886070 2921 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [term 1 LEADER]: Becoming Leader. State: Replica: 14e67237fdcb495ea3ce75cab580de76, State: Running, Role: LEADER
I20260808 00:37:23.886264 2921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } }
Aug 08 00:37:23 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:23.886726 2924 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e67237fdcb495ea3ce75cab580de76. Latest consensus state: current_term: 1 leader_uuid: "14e67237fdcb495ea3ce75cab580de76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } } }
I20260808 00:37:23.886816 2924 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:23.887151 2923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14e67237fdcb495ea3ce75cab580de76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e67237fdcb495ea3ce75cab580de76" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37291 } } }
I20260808 00:37:23.887218 2923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:23.887440 2938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:23.888573 2938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:23.890085 2938 catalog_manager.cc:1347] Generated new cluster ID: 9a7a07d893ba47eebeb88965be862854
I20260808 00:37:23.890129 2938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:23.891301 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:23.879081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.1.208.62:37291)
0808 00:37:23.879316 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:23.879329 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:23.879368 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:37:23.880014 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:23.880025 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:23.880098 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:23.880261 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:23.881007 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:23.882459 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:23.883389 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:23.884043 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:23.884154 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:23.887137 (+ 2983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:23.887142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:23.887142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:23.887155 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:23.889205 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:23.889664 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:23.889667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:23.889667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:23.889771 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:23.890012 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:23.890013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:23.890013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:23.890322 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:23.890865 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:23.890948 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 00:37:23.891095 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
I20260808 00:37:23.904441 2938 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:23.904958 2938 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:23.912675 2938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14e67237fdcb495ea3ce75cab580de76: Generated new TSK 0
I20260808 00:37:23.912891 2938 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:23.947790 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:24.030735 2962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:24.030946 2962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:24.030969 2962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:24.032117 2962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:24.032160 2962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:24.032186 2962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:24.033842 2962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:24.034088 2962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:24.034304 2962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:24.039215 2970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:24.039215 2972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:24.039499 2969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:24.040117 2962 server_base.cc:1034] running on GCE node
I20260808 00:37:24.040380 2962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:24.040618 2962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:24.041765 2962 hybrid_clock.cc:648] HybridClock initialized: now 1786149444041745 us; error 32 us; skew 500 ppm
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:24.044242 2962 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:24.044725 2962 webserver.cc:460] Webserver started at http://127.1.208.1:38629/ using document root <none> and password file <none>
I20260808 00:37:24.044929 2962 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:24.044976 2962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:24.045102 2962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:24.046212 2962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "bb8285d36762487f9cece1abf2290fd4"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "dc05baeb28ef85085aa2262b62932117"
server_key_iv: "852597c74fa681c3318628665f8329c8"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.046578 2962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "bb8285d36762487f9cece1abf2290fd4"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "dc05baeb28ef85085aa2262b62932117"
server_key_iv: "852597c74fa681c3318628665f8329c8"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.047897 2962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:24.048617 2981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.048766 2962 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:24.048862 2962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "bb8285d36762487f9cece1abf2290fd4"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "dc05baeb28ef85085aa2262b62932117"
server_key_iv: "852597c74fa681c3318628665f8329c8"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.048928 2962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:24.060503 2962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:24.062379 2962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:24.062513 2962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:24.062737 2962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:24.063071 2962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:24.063107 2962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.063135 2962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:24.063156 2962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.069037 2962 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:33847
I20260808 00:37:24.069079 3162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:33847 every 8 connection(s)
I20260808 00:37:24.069428 2962 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:24.073177 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2962
I20260808 00:37:24.073254 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:24.073459 1856 external_mini_cluster.cc:1428] Setting key f62f90c102c5af2270880c0148b90b3d
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:24.079535 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.070179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:42517 (local address 127.1.208.62:37291)
0808 00:37:24.070301 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.070304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.070445 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:37:24.071513 (+ 1068us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.071516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.071529 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.071582 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.072220 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.072735 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.073936 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.074215 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.074292 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.076475 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.076477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.076477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.076484 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.078163 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.078563 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.078564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.078565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.078640 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.078875 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.078876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.078876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.079023 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.079090 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.079321 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.079431 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 00:37:24.079644 3168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.070132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37291 (local address 127.1.208.1:42517)
0808 00:37:24.070318 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.070364 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.071403 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.071666 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.071671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.071933 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:24.072121 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.072126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.072818 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.072819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.073872 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.073876 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.074297 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.074298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.074402 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.074904 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:24.074912 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:24.076383 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:24.078256 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.078263 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.078265 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.078484 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.078712 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.078713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.078713 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.078817 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.079076 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:24.079079 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:24.079201 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.079281 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.079408 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260808 00:37:24.080263 3164 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37291
I20260808 00:37:24.080367 3164 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:24.080557 3164 heartbeater.cc:511] Master 127.1.208.62:37291 requested a full tablet report, sending...
I20260808 00:37:24.081198 2843 ts_manager.cc:194] Registered new tserver with Master: bb8285d36762487f9cece1abf2290fd4 (127.1.208.1:33847)
I20260808 00:37:24.082139 2843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:42517
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:24.103885 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:24.189105 3173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:24.189366 3173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:24.189397 3173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:24.190522 3173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:24.190555 3173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:24.190580 3173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:24.192035 3173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:24.192221 3173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:24.192412 3173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:24.197389 3185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:24.197489 3173 server_base.cc:1034] running on GCE node
W20260808 00:37:24.197423 3182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:24.197389 3183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:24.197902 3173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:24.198119 3173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:24.199247 3173 hybrid_clock.cc:648] HybridClock initialized: now 1786149444199243 us; error 33 us; skew 500 ppm
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:24.201720 3173 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:24.202185 3173 webserver.cc:460] Webserver started at http://127.1.208.2:36257/ using document root <none> and password file <none>
I20260808 00:37:24.202389 3173 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:24.202452 3173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:24.202582 3173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:24.203661 3173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "fad2f24545d94bd7a89949621382bd96"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "d0c8d575620234f721e1153a83066baf"
server_key_iv: "7a2410290860cfd9c65308b52cae1cac"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.204011 3173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "fad2f24545d94bd7a89949621382bd96"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "d0c8d575620234f721e1153a83066baf"
server_key_iv: "7a2410290860cfd9c65308b52cae1cac"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.205464 3173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:24.206178 3196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.206315 3173 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:24.206394 3173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "fad2f24545d94bd7a89949621382bd96"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "d0c8d575620234f721e1153a83066baf"
server_key_iv: "7a2410290860cfd9c65308b52cae1cac"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.206467 3173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:24.218739 3173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:24.220619 3173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:24.220742 3173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:24.220979 3173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:24.221336 3173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:24.221376 3173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.221400 3173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:24.221422 3173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.227414 3173 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:45183
I20260808 00:37:24.227468 3404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:45183 every 8 connection(s)
I20260808 00:37:24.227792 3173 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:24.229818 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3173
I20260808 00:37:24.229919 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:24.230105 1856 external_mini_cluster.cc:1428] Setting key fae2ff5f48281edd0bcb3f10a92c4185
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:24.238045 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.228504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:51479 (local address 127.1.208.62:37291)
0808 00:37:24.228588 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.228590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.228749 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:37:24.229827 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.229829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.229839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.229889 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.230689 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.231367 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.232385 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.232636 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.232691 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.235294 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.235296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.235296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.235304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.236677 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.237072 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.237074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.237074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.237132 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.237396 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.237397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.237397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.237523 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.237581 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.237857 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.237950 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:37:24.238217 3408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.228475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37291 (local address 127.1.208.2:51479)
0808 00:37:24.228629 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.228668 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.229708 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.229976 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.229981 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.230358 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:24.230571 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.230576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.231474 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.231475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.232294 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.232297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.232717 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.232718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.232868 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.233501 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:24.233513 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:24.235198 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:24.236754 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.236761 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.236764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.237004 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.237186 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.237187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.237187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.237337 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.237578 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:24.237585 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:24.237729 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.237813 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.237987 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260808 00:37:24.238646 3405 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37291
I20260808 00:37:24.238745 3405 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:24.238940 3405 heartbeater.cc:511] Master 127.1.208.62:37291 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
I20260808 00:37:24.239302 2843 ts_manager.cc:194] Registered new tserver with Master: fad2f24545d94bd7a89949621382bd96 (127.1.208.2:45183)
I20260808 00:37:24.239661 2843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:51479
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:24.256399 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:24.340158 3413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:24.340369 3413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:24.340391 3413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:24.341604 3413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:24.341647 3413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:24.341674 3413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:24.343217 3413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37291
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:24.343457 3413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:24.343680 3413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:24.348726 3422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:24.348806 3413 server_base.cc:1034] running on GCE node
W20260808 00:37:24.348711 3426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:24.348729 3428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:24.349232 3413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:24.349519 3413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:24.350658 3413 hybrid_clock.cc:648] HybridClock initialized: now 1786149444350645 us; error 25 us; skew 500 ppm
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:24.353900 3413 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:24.354383 3413 webserver.cc:460] Webserver started at http://127.1.208.3:41713/ using document root <none> and password file <none>
I20260808 00:37:24.354609 3413 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:24.354658 3413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:24.354789 3413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:24.355955 3413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "304e78bb1d8e272f72944defcbf93ba5"
server_key_iv: "941511206a7be5759db89ac49cb84e23"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.356362 3413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "304e78bb1d8e272f72944defcbf93ba5"
server_key_iv: "941511206a7be5759db89ac49cb84e23"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.358008 3413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:24.358795 3438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.358948 3413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:24.359028 3413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6"
format_stamp: "Formatted at 2026-08-08 00:37:24 on dist-test-slave-gjw7"
server_key: "304e78bb1d8e272f72944defcbf93ba5"
server_key_iv: "941511206a7be5759db89ac49cb84e23"
server_key_version: "encryptionkey@0"
I20260808 00:37:24.359092 3413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:24.372838 3413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:24.375356 3413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:24.375557 3413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:24.375969 3413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:24.376345 3413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:24.376382 3413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.376413 3413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:24.376430 3413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:24.394052 3413 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:35979
I20260808 00:37:24.394243 3639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:35979 every 8 connection(s)
I20260808 00:37:24.394516 3413 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:24.402261 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3413
I20260808 00:37:24.402352 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:24.402506 1856 external_mini_cluster.cc:1428] Setting key 1a64529137a40d0558be67c5e1d3118f
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:24.405793 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.395487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:59655 (local address 127.1.208.62:37291)
0808 00:37:24.395550 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.395552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.395875 (+ 323us) server_negotiation.cc:407] Connection header received
0808 00:37:24.397205 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.397206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.397217 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.397280 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.398206 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.398715 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.400007 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.400192 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.400239 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.403258 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.403260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.403260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.403268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.404433 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.404807 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.404808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.404808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.404856 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.405094 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.405096 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.405096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.405245 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.405387 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.405628 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.405720 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:37:24.406029 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.395468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37291 (local address 127.1.208.3:59655)
0808 00:37:24.395715 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.395773 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.397103 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.397387 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.397392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.397787 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:24.398132 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.398139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.398810 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.398811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.399938 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.399941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.400293 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.400294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.400454 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.401182 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:24.401192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:24.403167 (+ 1975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:24.404509 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.404517 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.404519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.404746 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.404900 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:24.404900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:24.404901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:24.405012 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:24.405382 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:24.405386 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:24.405522 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.405612 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.405742 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":82,"threads_started":1}
I20260808 00:37:24.406571 3640 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37291
I20260808 00:37:24.406684 3640 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:24.406890 3640 heartbeater.cc:511] Master 127.1.208.62:37291 requested a full tablet report, sending...
I20260808 00:37:24.407289 2843 ts_manager.cc:194] Registered new tserver with Master: c08a42c7f5ef4c04acb3bd464e3dd2f6 (127.1.208.3:35979)
I20260808 00:37:24.407696 2843 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:59655
I20260808 00:37:24.414161 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:37:24.421397 2915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.415814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.1.208.62:37291)
0808 00:37:24.415860 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.415862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.416129 (+ 267us) server_negotiation.cc:407] Connection header received
0808 00:37:24.416173 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.416174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.416183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.416229 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.416811 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.417318 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.417977 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.418154 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.418196 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.419270 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.419272 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.419272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.419278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.420465 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.420780 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.420780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.420781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.420827 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.420956 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.420956 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.420956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.421090 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.421171 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.421233 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.421327 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:24.423615 2843 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:24.425637 2843 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:24.433442 3661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:35979 (local address 127.0.0.1:34260)
0808 00:37:24.428857 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.428893 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.428964 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.429354 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.429358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.429361 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:24.429437 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.429440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.430733 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.430734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.432208 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.432210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.433097 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.433178 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.433194 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.433269 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.433349 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":54,"thread_start_us":65,"threads_started":1}
I20260808 00:37:24.433449 3663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34260 (local address 127.1.208.3:35979)
0808 00:37:24.428831 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.428837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.428955 (+ 118us) server_negotiation.cc:407] Connection header received
0808 00:37:24.429159 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.429163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.429181 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.429258 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:24.429569 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.430625 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.432297 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.433028 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433087 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.433133 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.433326 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.433368 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":162,"threads_started":1}
I20260808 00:37:24.434192 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:33847 (local address 127.0.0.1:58734)
0808 00:37:24.429086 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.429095 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.429140 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.429416 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.429417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.429419 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:24.429469 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.429471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.430728 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.430729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.432203 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.432206 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.433960 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.434024 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.434028 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.434077 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.434122 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":57,"thread_start_us":69,"threads_started":1}
I20260808 00:37:24.434229 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.1.208.1:33847)
0808 00:37:24.429192 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.429196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.429208 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:37:24.429280 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.429283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.429325 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.429400 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:24.429518 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.430648 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433166 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.433859 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433930 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.433991 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.434111 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.434154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":92,"threads_started":1}
I20260808 00:37:24.434438 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:45183 (local address 127.0.0.1:55688)
0808 00:37:24.428925 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.428935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.428991 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.430951 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.430952 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.430955 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:24.431014 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.431016 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.431942 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.431943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.432929 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.432933 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.434235 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.434236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.434301 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.434304 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.434350 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.434382 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":94,"threads_started":1}
I20260808 00:37:24.434635 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.428670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.1.208.2:45183)
0808 00:37:24.430811 (+ 2141us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.430815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.430823 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:24.430855 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.430857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.430868 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.430933 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:24.431100 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.431839 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.433018 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.434089 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.434145 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.434198 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.434504 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.434552 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2109,"thread_start_us":149,"threads_started":1}
I20260808 00:37:24.434588 3526 tablet_service.cc:1467] Processing CreateTablet for tablet c097c59b76c440ae92d765deeb796ccd (DEFAULT_TABLE table=test-table [id=0b56478d257e4b93b808e5030fce23fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:24.434978 3526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c097c59b76c440ae92d765deeb796ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:24.435115 3057 tablet_service.cc:1467] Processing CreateTablet for tablet c097c59b76c440ae92d765deeb796ccd (DEFAULT_TABLE table=test-table [id=0b56478d257e4b93b808e5030fce23fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:24.435374 3057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c097c59b76c440ae92d765deeb796ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:24.438201 3671 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: Bootstrap starting.
I20260808 00:37:24.438848 3673 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: Bootstrap starting.
I20260808 00:37:24.439049 3284 tablet_service.cc:1467] Processing CreateTablet for tablet c097c59b76c440ae92d765deeb796ccd (DEFAULT_TABLE table=test-table [id=0b56478d257e4b93b808e5030fce23fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:24.439393 3284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c097c59b76c440ae92d765deeb796ccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:24.439558 3673 tablet_bootstrap.cc:654] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:24.439951 3671 tablet_bootstrap.cc:654] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:24.440313 3673 log.cc:824] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:24.441231 3671 log.cc:824] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:24.442322 3671 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: No bootstrap required, opened a new log
I20260808 00:37:24.442407 3671 ts_tablet_manager.cc:1397] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: Time spent bootstrapping tablet: real 0.004s user 0.004s sys 0.000s
I20260808 00:37:24.442710 3673 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: No bootstrap required, opened a new log
I20260808 00:37:24.442850 3673 ts_tablet_manager.cc:1397] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 00:37:24.444950 3678 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Bootstrap starting.
I20260808 00:37:24.446588 3678 tablet_bootstrap.cc:654] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:24.446688 3671 raft_consensus.cc:348] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.446761 3673 raft_consensus.cc:348] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.446947 3671 raft_consensus.cc:374] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:24.447026 3671 raft_consensus.cc:729] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb8285d36762487f9cece1abf2290fd4, State: Initialized, Role: FOLLOWER
I20260808 00:37:24.447021 3673 raft_consensus.cc:374] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:24.447073 3673 raft_consensus.cc:729] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c08a42c7f5ef4c04acb3bd464e3dd2f6, State: Initialized, Role: FOLLOWER
I20260808 00:37:24.447180 3671 consensus_queue.cc:260] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.447204 3673 consensus_queue.cc:260] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.447645 3164 heartbeater.cc:503] Master 127.1.208.62:37291 was elected leader, sending a full tablet report...
I20260808 00:37:24.447659 3673 ts_tablet_manager.cc:1428] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260808 00:37:24.447685 3671 ts_tablet_manager.cc:1428] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 00:37:24.447817 3640 heartbeater.cc:503] Master 127.1.208.62:37291 was elected leader, sending a full tablet report...
I20260808 00:37:24.447870 3678 log.cc:824] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:24.448922 3678 tablet_bootstrap.cc:492] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: No bootstrap required, opened a new log
I20260808 00:37:24.448997 3678 ts_tablet_manager.cc:1397] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 00:37:24.452395 3678 raft_consensus.cc:348] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.452600 3678 raft_consensus.cc:374] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:24.452642 3678 raft_consensus.cc:729] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fad2f24545d94bd7a89949621382bd96, State: Initialized, Role: FOLLOWER
I20260808 00:37:24.452757 3678 consensus_queue.cc:260] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.453150 3405 heartbeater.cc:503] Master 127.1.208.62:37291 was elected leader, sending a full tablet report...
I20260808 00:37:24.453150 3678 ts_tablet_manager.cc:1428] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260808 00:37:24.454541 3680 raft_consensus.cc:482] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:24.454622 3680 raft_consensus.cc:504] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.455045 3680 leader_election.cc:288] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fad2f24545d94bd7a89949621382bd96 (127.1.208.2:45183), bb8285d36762487f9cece1abf2290fd4 (127.1.208.1:33847)
I20260808 00:37:24.458047 3644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.455197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:45183 (local address 127.1.208.3:35965)
0808 00:37:24.455253 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.455266 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.455314 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.455489 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.455491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.455493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:24.455690 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.455693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.456320 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.456321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.457210 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.457213 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.457759 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.457760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.457879 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.457883 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.457937 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.457975 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:37:24.458047 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.455254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35965 (local address 127.1.208.2:45183)
0808 00:37:24.455317 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.455319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.455349 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:37:24.455373 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.455374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.455386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.455434 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:24.455766 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.456250 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.457318 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.457691 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.457750 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.457761 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.457950 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.457986 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:37:24.458369 3325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c097c59b76c440ae92d765deeb796ccd" candidate_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fad2f24545d94bd7a89949621382bd96" is_pre_election: true
I20260808 00:37:24.458501 3325 raft_consensus.cc:2454] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08a42c7f5ef4c04acb3bd464e3dd2f6 in term 0.
I20260808 00:37:24.458745 3450 leader_election.cc:302] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08a42c7f5ef4c04acb3bd464e3dd2f6, fad2f24545d94bd7a89949621382bd96; no voters:
I20260808 00:37:24.458848 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.455450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55571 (local address 127.1.208.1:33847)
0808 00:37:24.455489 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.455491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.455739 (+ 248us) server_negotiation.cc:407] Connection header received
0808 00:37:24.455816 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.455817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.455829 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.455896 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:24.456167 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.456630 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.457907 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.458375 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.458440 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.458454 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.458744 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.458782 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:24.458905 3687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.455399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:33847 (local address 127.1.208.3:55571)
0808 00:37:24.455674 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:24.455683 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:24.455744 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:24.455901 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:24.455901 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:24.455903 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:24.456104 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.456106 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.456702 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.456704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.457827 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:24.457830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.458559 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.458561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:24.458672 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.458676 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:24.458727 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:37:24.458765 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":154,"threads_started":1}
I20260808 00:37:24.458911 3680 raft_consensus.cc:2781] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:24.458997 3680 raft_consensus.cc:482] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:24.459028 3680 raft_consensus.cc:3037] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:24.459436 3088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c097c59b76c440ae92d765deeb796ccd" candidate_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8285d36762487f9cece1abf2290fd4" is_pre_election: true
I20260808 00:37:24.459591 3088 raft_consensus.cc:2454] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c08a42c7f5ef4c04acb3bd464e3dd2f6 in term 0.
I20260808 00:37:24.460536 3680 raft_consensus.cc:504] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.460645 3680 leader_election.cc:288] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [CANDIDATE]: Term 1 election: Requested vote from peers fad2f24545d94bd7a89949621382bd96 (127.1.208.2:45183), bb8285d36762487f9cece1abf2290fd4 (127.1.208.1:33847)
I20260808 00:37:24.460791 3325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c097c59b76c440ae92d765deeb796ccd" candidate_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fad2f24545d94bd7a89949621382bd96"
I20260808 00:37:24.460819 3088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c097c59b76c440ae92d765deeb796ccd" candidate_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb8285d36762487f9cece1abf2290fd4"
I20260808 00:37:24.460871 3325 raft_consensus.cc:3037] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:24.460877 3088 raft_consensus.cc:3037] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:24.462728 3088 raft_consensus.cc:2454] T c097c59b76c440ae92d765deeb796ccd P bb8285d36762487f9cece1abf2290fd4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08a42c7f5ef4c04acb3bd464e3dd2f6 in term 1.
I20260808 00:37:24.462728 3325 raft_consensus.cc:2454] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c08a42c7f5ef4c04acb3bd464e3dd2f6 in term 1.
I20260808 00:37:24.462970 3450 leader_election.cc:302] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c08a42c7f5ef4c04acb3bd464e3dd2f6, fad2f24545d94bd7a89949621382bd96; no voters:
I20260808 00:37:24.463081 3680 raft_consensus.cc:2781] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:24.463397 3680 raft_consensus.cc:686] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [term 1 LEADER]: Becoming Leader. State: Replica: c08a42c7f5ef4c04acb3bd464e3dd2f6, State: Running, Role: LEADER
I20260808 00:37:24.463515 3680 consensus_queue.cc:237] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } }
I20260808 00:37:24.464434 2843 catalog_manager.cc:5495] T c097c59b76c440ae92d765deeb796ccd P c08a42c7f5ef4c04acb3bd464e3dd2f6 reported cstate change: term changed from 0 to 1, leader changed from <none> to c08a42c7f5ef4c04acb3bd464e3dd2f6 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c08a42c7f5ef4c04acb3bd464e3dd2f6" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fad2f24545d94bd7a89949621382bd96" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 45183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb8285d36762487f9cece1abf2290fd4" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33847 } health_report { overall_health: UNKNOWN } } }
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
W20260808 00:37:24.478394 3406 tablet.cc:2369] T c097c59b76c440ae92d765deeb796ccd P fad2f24545d94bd7a89949621382bd96: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:24.478544 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.470051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58750 (local address 127.1.208.1:33847)
0808 00:37:24.470100 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.470102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.470362 (+ 260us) server_negotiation.cc:407] Connection header received
0808 00:37:24.470393 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.470394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.470405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.470454 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.471141 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.471656 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.472497 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.472689 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.472740 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.475552 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.475566 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.475566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.475574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.477279 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.477646 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.477650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.477650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.477713 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.477895 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.477896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.477897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.478063 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.478195 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.478351 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.478450 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:24.504689 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.498180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.1.208.1:33847)
0808 00:37:24.498243 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.498245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.498307 (+ 62us) server_negotiation.cc:407] Connection header received
0808 00:37:24.498371 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.498372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.498382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.498432 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.498977 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.499479 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.500143 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.500321 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.500395 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.502727 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.502729 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.502729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.502736 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.503805 (+ 1069us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.504092 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.504092 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.504093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.504137 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.504262 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.504263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.504263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.504347 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.504429 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.504557 (+ 128us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.504619 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[2763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149444, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:24.530555 3667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:24.523838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58780 (local address 127.1.208.1:33847)
0808 00:37:24.523912 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:24.523914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:24.524016 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:37:24.524101 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:24.524101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:24.524111 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:24.524152 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:24.524690 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.525154 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.525935 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:24.526118 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:24.526174 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:24.528456 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:24.528458 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:24.528458 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:24.528466 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:24.529695 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.529991 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.529991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.529992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.530041 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:24.530185 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:24.530185 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:24.530185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:24.530268 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:24.530333 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:24.530427 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 00:37:24.530484 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260808 00:37:24.531138 3022 tablet_service.cc:2908] Rejecting scan request for tablet c097c59b76c440ae92d765deeb796ccd: Uninitialized: clean time has not yet been initialized
I20260808 00:37:24.532083 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2962
I20260808 00:37:24.546224 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3173
I20260808 00:37:24.595415 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3413
I20260808 00:37:24.610688 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2792
2026-08-08T00:37:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2181 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[3711](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[3711](info): set up 2 sockets
Aug 08 00:37:24 dist-test-slave-gjw7 krb5kdc[3711](info): commencing operation
krb5kdc: starting...
W20260808 00:37:25.795162 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:25 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:25.915584 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:33881
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:33881
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:25.996389 3736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:25.996587 3736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:25.996615 3736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:25.997764 3736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:25.997802 3736 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:37:25.997817 3736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:25.997834 3736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:25.997849 3736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:25.997865 3736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:25.999349 3736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:33881
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:33881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:25.999598 3736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:25.999809 3736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:26.004472 3743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.004546 3736 server_base.cc:1034] running on GCE node
W20260808 00:37:26.004453 3742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:26.004454 3745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.004987 3736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:26.005273 3736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:26.006439 3736 hybrid_clock.cc:648] HybridClock initialized: now 1786149446006431 us; error 34 us; skew 500 ppm
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:26.013917 3736 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:26.014366 3736 webserver.cc:460] Webserver started at http://127.1.208.62:41849/ using document root <none> and password file <none>
I20260808 00:37:26.014575 3736 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:26.014617 3736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:26.014710 3736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:26.015730 3736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "2072b6dd1c914cc493c5edfd320a5a12"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "bf9873459cb71dec42e18f7820cecc13"
server_key_iv: "511ad4531d5e06ee4064cd97c22d91a9"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.016042 3736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "2072b6dd1c914cc493c5edfd320a5a12"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "bf9873459cb71dec42e18f7820cecc13"
server_key_iv: "511ad4531d5e06ee4064cd97c22d91a9"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.017387 3736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:26.018055 3753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.018213 3736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:26.018298 3736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "2072b6dd1c914cc493c5edfd320a5a12"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "bf9873459cb71dec42e18f7820cecc13"
server_key_iv: "511ad4531d5e06ee4064cd97c22d91a9"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.018383 3736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:26.033499 3736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:26.035275 3736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:26.035406 3736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:26.040014 3736 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:33881
I20260808 00:37:26.040091 3850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:33881 every 8 connection(s)
I20260808 00:37:26.040410 3736 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:26.041132 3853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.043561 3853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Bootstrap starting.
I20260808 00:37:26.044162 3853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.044440 3853 log.cc:824] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:26.045095 3853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: No bootstrap required, opened a new log
I20260808 00:37:26.047355 3853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } }
I20260808 00:37:26.047497 3853 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.047529 3853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2072b6dd1c914cc493c5edfd320a5a12, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.047672 3853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } }
I20260808 00:37:26.047753 3853 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:26.047784 3853 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:26.047842 3853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.048352 3853 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } }
I20260808 00:37:26.048451 3853 leader_election.cc:302] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2072b6dd1c914cc493c5edfd320a5a12; no voters:
I20260808 00:37:26.048607 3853 leader_election.cc:288] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:26.048650 3856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:26.048830 3856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [term 1 LEADER]: Becoming Leader. State: Replica: 2072b6dd1c914cc493c5edfd320a5a12, State: Running, Role: LEADER
I20260808 00:37:26.048868 3853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:26.048924 3856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } }
I20260808 00:37:26.049315 3859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2072b6dd1c914cc493c5edfd320a5a12. Latest consensus state: current_term: 1 leader_uuid: "2072b6dd1c914cc493c5edfd320a5a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } } }
I20260808 00:37:26.049389 3859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:26.049276 3858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2072b6dd1c914cc493c5edfd320a5a12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2072b6dd1c914cc493c5edfd320a5a12" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33881 } } }
I20260808 00:37:26.049530 3858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:26.049602 3865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:26.050107 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3736
I20260808 00:37:26.050158 3865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:26.050199 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:26.050359 1856 external_mini_cluster.cc:1428] Setting key 95b2596fb69d37c668cba5520ae4e639
I20260808 00:37:26.051935 3865 catalog_manager.cc:1347] Generated new cluster ID: 34189682c3214562903453d057cd62ab
I20260808 00:37:26.051985 3865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.061957 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.050822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.1.208.62:33881)
0808 00:37:26.051193 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.051203 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.051231 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:37:26.051357 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.051365 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.051421 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.051567 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.052341 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.053729 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.054786 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.055369 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.055463 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.058277 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.058282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.058282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.058296 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.060243 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.060630 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.060634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.060634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.060763 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.060938 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.060939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.060939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.061306 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.061472 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.061540 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.061703 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":80,"threads_started":1}
I20260808 00:37:26.064635 3865 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:26.065148 3865 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:26.073803 3865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Generated new TSK 0
I20260808 00:37:26.073988 3865 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:26.099917 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33881
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:26.182394 3911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:26.182608 3911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:26.182631 3911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:26.183804 3911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:26.183847 3911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:26.183867 3911 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:26.183883 3911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:26.185456 3911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:26.185684 3911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:26.185913 3911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:26.190868 3923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:26.190806 3927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:26.190814 3920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.191056 3911 server_base.cc:1034] running on GCE node
I20260808 00:37:26.191269 3911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:26.191501 3911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:26.192643 3911 hybrid_clock.cc:648] HybridClock initialized: now 1786149446192629 us; error 28 us; skew 500 ppm
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:26.195199 3911 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:26.195617 3911 webserver.cc:460] Webserver started at http://127.1.208.1:34369/ using document root <none> and password file <none>
I20260808 00:37:26.195820 3911 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:26.195873 3911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:26.196012 3911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:26.197180 3911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "07498aba7d3b426487a8d96c3af5fb29"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "1cd84d421b14952ddce39f5688e4e8fd"
server_key_iv: "cc752b4f939dc00d38d5f1e1013130bd"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.197561 3911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "07498aba7d3b426487a8d96c3af5fb29"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "1cd84d421b14952ddce39f5688e4e8fd"
server_key_iv: "cc752b4f939dc00d38d5f1e1013130bd"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.198818 3911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:26.199486 3938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.199628 3911 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:26.199709 3911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "07498aba7d3b426487a8d96c3af5fb29"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "1cd84d421b14952ddce39f5688e4e8fd"
server_key_iv: "cc752b4f939dc00d38d5f1e1013130bd"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.199772 3911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:26.214170 3911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:26.215955 3911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:26.216079 3911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:26.216570 3911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:26.216609 3911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.216639 3911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:26.216672 3911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.223166 3911 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:37043
I20260808 00:37:26.223321 4139 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:37043 every 8 connection(s)
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.223587 3911 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:26.224776 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3911
I20260808 00:37:26.224864 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:26.225028 1856 external_mini_cluster.cc:1428] Setting key 36f26768313ebf07f6c9b57ca2cec2d7
I20260808 00:37:26.226687 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.217152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:55597 (local address 127.1.208.62:33881)
0808 00:37:26.217201 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.217203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.217494 (+ 291us) server_negotiation.cc:407] Connection header received
0808 00:37:26.218035 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.218036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.218049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.218101 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.218924 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.219710 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.221194 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.221493 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.221556 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.224049 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.224050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.224051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.224059 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.225251 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.225784 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.225785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.225786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.225872 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.226093 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.226094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.226094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.226221 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.226290 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.226506 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.226603 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:26.226761 3948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.217158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.1:55597)
0808 00:37:26.217365 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.217418 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.217931 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.218173 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.218179 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.218536 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.218807 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.218814 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.219803 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.219804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.221098 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.221102 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.221575 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.221576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.221706 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.222214 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.222224 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.223944 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.225424 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.225433 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.225435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.225716 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.225929 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.225930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.225930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.226039 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.226282 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.226285 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.226393 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.226472 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.226596 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":108,"threads_started":1}
I20260808 00:37:26.227253 4141 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33881
I20260808 00:37:26.227337 4141 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:26.227535 4141 heartbeater.cc:511] Master 127.1.208.62:33881 requested a full tablet report, sending...
I20260808 00:37:26.228101 3778 ts_manager.cc:194] Registered new tserver with Master: 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043)
I20260808 00:37:26.228973 3778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:55597
I20260808 00:37:26.234680 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.228459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:50165 (local address 127.1.208.62:33881)
0808 00:37:26.228497 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.228499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.229312 (+ 813us) server_negotiation.cc:407] Connection header received
0808 00:37:26.229410 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.229411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.229425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.229482 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.229894 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.230349 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.231267 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.231470 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.231516 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.232442 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.232443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.232444 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.232449 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.233681 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.234032 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.234033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.234033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.234080 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.234261 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.234262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.234262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.234355 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.234399 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.234555 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.234619 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:37:26.234738 4149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.228382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.1:50165)
0808 00:37:26.229221 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.229234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.229316 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.229488 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.229489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.229685 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.229815 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.229818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.230446 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.230447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.231183 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.231187 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.231531 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.231532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.231605 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.231957 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.231962 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.232355 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.233754 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.233755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.233756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.233963 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.234131 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.234132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.234132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.234200 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.234407 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.234408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.234466 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.234536 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.234635 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:26.251935 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33881
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:26.337514 4152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:26.337710 4152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:26.337740 4152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:26.338869 4152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:26.338912 4152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:26.338940 4152 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:26.338964 4152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:26.340425 4152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:26.340652 4152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:26.340864 4152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:26.345589 4167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.345618 4152 server_base.cc:1034] running on GCE node
W20260808 00:37:26.345657 4164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:26.345590 4160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.345968 4152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:26.346196 4152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:26.347347 4152 hybrid_clock.cc:648] HybridClock initialized: now 1786149446347329 us; error 31 us; skew 500 ppm
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:26.349933 4152 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:26.350433 4152 webserver.cc:460] Webserver started at http://127.1.208.2:46707/ using document root <none> and password file <none>
I20260808 00:37:26.350636 4152 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:26.350682 4152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:26.350812 4152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:26.351848 4152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "11028bd5d2fe4e1d9289713278212b2f"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "78105e21e28a4ff4464779f0012fe690"
server_key_iv: "0e6897ba614cdbd12421a904b7607dc2"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.352187 4152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "11028bd5d2fe4e1d9289713278212b2f"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "78105e21e28a4ff4464779f0012fe690"
server_key_iv: "0e6897ba614cdbd12421a904b7607dc2"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.353672 4152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:26.354331 4181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.354481 4152 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:26.354564 4152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "11028bd5d2fe4e1d9289713278212b2f"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "78105e21e28a4ff4464779f0012fe690"
server_key_iv: "0e6897ba614cdbd12421a904b7607dc2"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.354624 4152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:26.361771 4152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:26.363523 4152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:26.363647 4152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:26.364142 4152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:26.364182 4152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.364209 4152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:26.364238 4152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.371480 4152 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:36077
I20260808 00:37:26.371538 4382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:36077 every 8 connection(s)
I20260808 00:37:26.371866 4152 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:26.373059 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.364549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:51495 (local address 127.1.208.62:33881)
0808 00:37:26.364611 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.364613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.364856 (+ 243us) server_negotiation.cc:407] Connection header received
0808 00:37:26.365546 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.365547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.365557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.365592 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.366422 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.366919 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.367950 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.368145 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.368200 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.370297 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.370298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.370299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.370307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.371841 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.372265 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.372266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.372266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.372339 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.372583 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.372584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.372584 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.372686 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.372737 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.372928 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.372994 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:37:26.373224 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.364533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.2:51495)
0808 00:37:26.364710 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.364758 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.365435 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.365675 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.365679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.366039 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.366322 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.366328 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.367008 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.367009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.367862 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.367865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.368219 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.368220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.368333 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.368745 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.368753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.370198 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.371927 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.371930 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.371931 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.372186 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.372392 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.372393 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.372393 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.372532 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.372733 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.372736 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.372841 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.372915 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.373037 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260808 00:37:26.373649 4383 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33881
I20260808 00:37:26.373734 4383 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:26.373917 4383 heartbeater.cc:511] Master 127.1.208.62:33881 requested a full tablet report, sending...
I20260808 00:37:26.374298 3776 ts_manager.cc:194] Registered new tserver with Master: 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077)
I20260808 00:37:26.374755 3776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:51495
I20260808 00:37:26.377482 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4152
I20260808 00:37:26.377576 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:26.377746 1856 external_mini_cluster.cc:1428] Setting key 523a740bc8a065de6c6d53da2b05ccba
I20260808 00:37:26.379824 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.374724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:33981 (local address 127.1.208.62:33881)
0808 00:37:26.374893 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.374895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.374903 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:26.374921 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.374922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.374931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.374986 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.375339 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.375785 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.376438 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.376614 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.376654 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.377517 (+ 863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.377518 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.377518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.377523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.378835 (+ 1312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.379181 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.379182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.379182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.379230 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.379390 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.379391 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.379391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.379517 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.379566 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.379709 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.379762 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260808 00:37:26.379879 4396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.374662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.2:33981)
0808 00:37:26.374791 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.374801 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.374845 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.374999 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.374999 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.375191 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.375263 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.375266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.375884 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.375885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.376379 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.376381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.376659 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.376660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.376712 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.377071 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.377076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.377444 (+ 368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.378919 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.378920 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.378920 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.379114 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.379279 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.379280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.379280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.379340 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.379569 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.379570 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.379613 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.379693 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.379786 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":6,"thread_start_us":76,"threads_started":1}
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:26.404228 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33881
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:26.486390 4399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:26.486575 4399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:26.486606 4399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:26.487723 4399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:26.487761 4399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:26.487788 4399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:26.487810 4399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:26.489212 4399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:33881
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:26.489467 4399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:26.489670 4399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:26.494604 4407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.494637 4399 server_base.cc:1034] running on GCE node
W20260808 00:37:26.494594 4406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:26.494598 4415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:26.494980 4399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:26.495206 4399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:26.496368 4399 hybrid_clock.cc:648] HybridClock initialized: now 1786149446496356 us; error 27 us; skew 500 ppm
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:26.498963 4399 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:26.499393 4399 webserver.cc:460] Webserver started at http://127.1.208.3:38749/ using document root <none> and password file <none>
I20260808 00:37:26.499610 4399 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:26.499660 4399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:26.499811 4399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:26.500994 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "d327b67104cb467fae3f3469723d9b85"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "b2e46164b1968c10cb895d752fd47ef0"
server_key_iv: "92e3d71a25755dc7f5272b7baf85c77c"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.501438 4399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "d327b67104cb467fae3f3469723d9b85"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "b2e46164b1968c10cb895d752fd47ef0"
server_key_iv: "92e3d71a25755dc7f5272b7baf85c77c"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.502831 4399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:26.503531 4425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.503693 4399 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:26.503780 4399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "d327b67104cb467fae3f3469723d9b85"
format_stamp: "Formatted at 2026-08-08 00:37:26 on dist-test-slave-gjw7"
server_key: "b2e46164b1968c10cb895d752fd47ef0"
server_key_iv: "92e3d71a25755dc7f5272b7baf85c77c"
server_key_version: "encryptionkey@0"
I20260808 00:37:26.503854 4399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:26.532753 4399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:26.535063 4399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:26.535214 4399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:26.535739 4399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:26.535785 4399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.535818 4399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:26.535845 4399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.545145 4399 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:37839
I20260808 00:37:26.545210 4636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:37839 every 8 connection(s)
I20260808 00:37:26.545549 4399 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:26.546284 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.536212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.3:52937)
0808 00:37:26.536405 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.536455 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.537124 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.537439 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.537444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.537798 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.538093 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.538100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.539107 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.539109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.540383 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.540387 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.540912 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.540914 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.541075 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.541613 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.541623 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.543122 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.544909 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.544914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.544916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.545158 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.545398 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.545399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.545399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.545503 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.545734 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.545741 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.545892 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.545998 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.546131 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260808 00:37:26.546607 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.536226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:52937 (local address 127.1.208.62:33881)
0808 00:37:26.536288 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.536290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.536556 (+ 266us) server_negotiation.cc:407] Connection header received
0808 00:37:26.537260 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.537262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.537273 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.537349 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.538229 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.538990 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.540478 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.540780 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.540854 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.543213 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.543215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.543215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.543223 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.544823 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.545250 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.545251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.545251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.545344 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.545561 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.545562 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.545563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.545679 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.545735 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.546440 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.546519 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:37:26.547025 4637 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33881
I20260808 00:37:26.547108 4637 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:26.547334 4637 heartbeater.cc:511] Master 127.1.208.62:33881 requested a full tablet report, sending...
I20260808 00:37:26.547717 3778 ts_manager.cc:194] Registered new tserver with Master: d327b67104cb467fae3f3469723d9b85 (127.1.208.3:37839)
I20260808 00:37:26.548185 3778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:52937
I20260808 00:37:26.548987 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4399
I20260808 00:37:26.549089 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:26.549319 1856 external_mini_cluster.cc:1428] Setting key 98ce4b4e9bbca63ae1a3775f05fe54da
I20260808 00:37:26.550232 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:37:26.554571 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.548047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35585 (local address 127.1.208.62:33881)
0808 00:37:26.548083 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.548084 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.548206 (+ 122us) server_negotiation.cc:407] Connection header received
0808 00:37:26.548277 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.548278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.548288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.548377 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.548950 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.549617 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.550622 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.550826 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.550879 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.551921 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.551922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.551922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.551929 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.553471 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.553797 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.553798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.553798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.553862 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.554038 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.554039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.554039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.554172 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.554244 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.554419 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.554490 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:37:26.554571 4648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.548002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.1.208.3:35585)
0808 00:37:26.548135 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.548145 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.548213 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.548387 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.548388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.548605 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.548732 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.548735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.549864 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.549864 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.550547 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.550550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.550883 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.550884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.551074 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.551551 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.551555 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.551844 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.553544 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.553545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.553546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.553726 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.553930 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.553930 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.553930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.553985 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.554235 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.554236 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.554274 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.554382 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.554466 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.576380 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.569880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.1.208.62:33881)
0808 00:37:26.569936 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.569938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.569945 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:26.570004 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.570005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.570015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.570064 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.570645 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.571125 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.571780 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.571949 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.571987 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.574304 (+ 2317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.574306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.574306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.574314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.575558 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.575832 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.575833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.575833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.575877 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.576012 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.576012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.576012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.576103 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.576168 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.576234 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.576312 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:37:26.577971 3778 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:26.579674 3778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:26.586246 4667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.581907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:37839 (local address 127.0.0.1:48010)
0808 00:37:26.582085 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.582125 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.582191 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.582728 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.582731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.582733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:26.582798 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.582800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.583834 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.583835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.584859 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.584862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.585947 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.585948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.586038 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.586052 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.586116 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.586168 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":42,"threads_started":1}
I20260808 00:37:26.586452 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.582263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48010 (local address 127.1.208.3:37839)
0808 00:37:26.582585 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.582588 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.582597 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:26.582634 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.582636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.582649 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.582722 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:26.582879 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.583762 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.584963 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.585871 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.585933 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.585999 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.586139 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.586180 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":98,"threads_started":1}
I20260808 00:37:26.586830 4668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.582098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36077 (local address 127.0.0.1:57524)
0808 00:37:26.582809 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.582818 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.582857 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.583151 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.583152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.583155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:26.583240 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.583241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.584507 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.584508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.585596 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.585600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.586594 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.586595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.586653 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.586658 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.586702 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.586757 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":84,"threads_started":1}
I20260808 00:37:26.587080 4670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.581963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37043 (local address 127.0.0.1:51954)
0808 00:37:26.582261 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.582272 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.582332 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.582887 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.582888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.582890 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:26.582922 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.582922 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.584473 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.584474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.585491 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.585494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.586863 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.586865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.586927 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.586931 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.586975 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.587012 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":38,"thread_start_us":149,"threads_started":1}
I20260808 00:37:26.587250 4515 tablet_service.cc:1467] Processing CreateTablet for tablet ca392ffec52242448c95ad9c371f0b18 (DEFAULT_TABLE table=test-table [id=260e227c0fe24f2286c3aef6ba7e5598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.587447 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.582643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57524 (local address 127.1.208.2:36077)
0808 00:37:26.582984 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.582988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.582998 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:26.583036 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.583038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.583055 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.583136 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:26.583351 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.584414 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.585679 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.586508 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.586576 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.586632 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.587159 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.587207 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":147,"threads_started":1}
I20260808 00:37:26.587667 4515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca392ffec52242448c95ad9c371f0b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.587913 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.582155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.1.208.1:37043)
0808 00:37:26.582485 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.582490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.582501 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:26.582579 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.582583 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.582600 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.582683 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:26.583030 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.584289 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.585566 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.586696 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.586773 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.586828 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.587260 (+ 432us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.587305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":189,"threads_started":1}
I20260808 00:37:26.588171 4266 tablet_service.cc:1467] Processing CreateTablet for tablet ca392ffec52242448c95ad9c371f0b18 (DEFAULT_TABLE table=test-table [id=260e227c0fe24f2286c3aef6ba7e5598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.588480 4266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca392ffec52242448c95ad9c371f0b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.591660 4676 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Bootstrap starting.
I20260808 00:37:26.592085 4677 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Bootstrap starting.
I20260808 00:37:26.592468 4013 tablet_service.cc:1467] Processing CreateTablet for tablet ca392ffec52242448c95ad9c371f0b18 (DEFAULT_TABLE table=test-table [id=260e227c0fe24f2286c3aef6ba7e5598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.592815 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca392ffec52242448c95ad9c371f0b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.593283 4676 tablet_bootstrap.cc:654] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.594115 4677 tablet_bootstrap.cc:654] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.594467 4676 log.cc:824] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:26.595182 4677 log.cc:824] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:26.595504 4676 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: No bootstrap required, opened a new log
I20260808 00:37:26.595574 4676 ts_tablet_manager.cc:1397] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 00:37:26.596585 4677 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: No bootstrap required, opened a new log
I20260808 00:37:26.596632 4677 ts_tablet_manager.cc:1397] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 00:37:26.598421 4681 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Bootstrap starting.
I20260808 00:37:26.599761 4676 raft_consensus.cc:348] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.599895 4676 raft_consensus.cc:374] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.599925 4676 raft_consensus.cc:729] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11028bd5d2fe4e1d9289713278212b2f, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.599967 4681 tablet_bootstrap.cc:654] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.600155 4676 consensus_queue.cc:260] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.600596 4676 ts_tablet_manager.cc:1428] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20260808 00:37:26.600718 4383 heartbeater.cc:503] Master 127.1.208.62:33881 was elected leader, sending a full tablet report...
I20260808 00:37:26.600634 4677 raft_consensus.cc:348] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.600792 4677 raft_consensus.cc:374] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.600838 4677 raft_consensus.cc:729] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d327b67104cb467fae3f3469723d9b85, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.600973 4677 consensus_queue.cc:260] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.601073 4681 log.cc:824] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:26.601387 4677 ts_tablet_manager.cc:1428] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 00:37:26.601526 4637 heartbeater.cc:503] Master 127.1.208.62:33881 was elected leader, sending a full tablet report...
I20260808 00:37:26.602391 4681 tablet_bootstrap.cc:492] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: No bootstrap required, opened a new log
I20260808 00:37:26.602454 4681 ts_tablet_manager.cc:1397] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 00:37:26.605839 4681 raft_consensus.cc:348] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.605958 4681 raft_consensus.cc:374] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.605988 4681 raft_consensus.cc:729] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07498aba7d3b426487a8d96c3af5fb29, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.606088 4681 consensus_queue.cc:260] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.606420 4681 ts_tablet_manager.cc:1428] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260808 00:37:26.606529 4141 heartbeater.cc:503] Master 127.1.208.62:33881 was elected leader, sending a full tablet report...
W20260808 00:37:26.622488 4384 tablet.cc:2369] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:37:26.724424 4142 tablet.cc:2369] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:37:26.796277 4640 tablet.cc:2369] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:26.824970 4684 raft_consensus.cc:482] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:26.825124 4684 raft_consensus.cc:504] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.825687 4684 leader_election.cc:288] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043), 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077)
I20260808 00:37:26.829428 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.826122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:53625 (local address 127.1.208.2:36077)
0808 00:37:26.826182 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.826184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.826192 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:26.826219 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.826220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.826229 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.826283 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:26.826559 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.827071 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.828215 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.828866 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.828945 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.828958 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.829315 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.829372 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:37:26.829432 4443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.825892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36077 (local address 127.1.208.3:53625)
0808 00:37:26.825951 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.825963 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.826009 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.826299 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.826301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.826303 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:26.826487 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.826489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.827178 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.827180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.828103 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.828106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.828959 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.828961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.829204 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.829211 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.829277 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.829347 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:37:26.829957 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca392ffec52242448c95ad9c371f0b18" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f" is_pre_election: true
I20260808 00:37:26.830083 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.825837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55859 (local address 127.1.208.1:37043)
0808 00:37:26.825890 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.825892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.826317 (+ 425us) server_negotiation.cc:407] Connection header received
0808 00:37:26.826363 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.826364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.826378 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.826430 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:26.826760 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.827440 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.828971 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.829566 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.829639 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.829650 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.829990 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.830027 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:37:26.830147 4295 raft_consensus.cc:2454] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 0.
I20260808 00:37:26.830169 4687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.825915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37043 (local address 127.1.208.3:55859)
0808 00:37:26.826236 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.826245 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.826308 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.826448 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.826449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.826450 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:26.826669 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.826681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.827525 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.827526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.828885 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.828889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.829716 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.829717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.829902 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.829907 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.829972 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.830013 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":183,"threads_started":1}
I20260808 00:37:26.830446 4432 leader_election.cc:302] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11028bd5d2fe4e1d9289713278212b2f, d327b67104cb467fae3f3469723d9b85; no voters:
I20260808 00:37:26.830538 4053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca392ffec52242448c95ad9c371f0b18" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07498aba7d3b426487a8d96c3af5fb29" is_pre_election: true
I20260808 00:37:26.830591 4684 raft_consensus.cc:2781] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:26.830680 4053 raft_consensus.cc:2454] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 0.
I20260808 00:37:26.830689 4684 raft_consensus.cc:482] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:26.830722 4684 raft_consensus.cc:3037] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.832440 4684 raft_consensus.cc:504] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.832590 4684 leader_election.cc:288] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 election: Requested vote from peers 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043), 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077)
I20260808 00:37:26.832759 4053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca392ffec52242448c95ad9c371f0b18" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07498aba7d3b426487a8d96c3af5fb29"
I20260808 00:37:26.832767 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca392ffec52242448c95ad9c371f0b18" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f"
I20260808 00:37:26.832839 4053 raft_consensus.cc:3037] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.832840 4295 raft_consensus.cc:3037] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.833937 4053 raft_consensus.cc:2454] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 1.
I20260808 00:37:26.834192 4426 leader_election.cc:302] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07498aba7d3b426487a8d96c3af5fb29, d327b67104cb467fae3f3469723d9b85; no voters:
I20260808 00:37:26.834301 4684 raft_consensus.cc:2781] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:26.834311 4295 raft_consensus.cc:2454] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 1.
I20260808 00:37:26.834471 4684 raft_consensus.cc:686] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 1 LEADER]: Becoming Leader. State: Replica: d327b67104cb467fae3f3469723d9b85, State: Running, Role: LEADER
I20260808 00:37:26.834579 4684 consensus_queue.cc:237] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } }
I20260808 00:37:26.835462 3778 catalog_manager.cc:5495] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 reported cstate change: term changed from 0 to 1, leader changed from <none> to d327b67104cb467fae3f3469723d9b85 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "d327b67104cb467fae3f3469723d9b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:26.871167 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.868532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48020 (local address 127.1.208.3:37839)
0808 00:37:26.868601 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.868603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.868613 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:26.868732 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.868733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.868759 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.868814 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:26.869011 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.869551 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.870313 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.870503 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.870558 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.870846 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:26.870946 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:26.871016 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.871063 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.871087 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:37:26.873560 4295 raft_consensus.cc:1261] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Refusing update from remote peer d327b67104cb467fae3f3469723d9b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:26.873587 4053 raft_consensus.cc:1261] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Refusing update from remote peer d327b67104cb467fae3f3469723d9b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:26.873946 4684 consensus_queue.cc:1035] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:26.874155 4688 consensus_queue.cc:1035] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:26.880692 3776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:26.880813 3776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:26.881661 3776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Sending DeleteTablet for 3 replicas of tablet ca392ffec52242448c95ad9c371f0b18
I20260808 00:37:26.882122 4515 tablet_service.cc:1514] Processing DeleteTablet for tablet ca392ffec52242448c95ad9c371f0b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:26 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:48010
I20260808 00:37:26.882135 4013 tablet_service.cc:1514] Processing DeleteTablet for tablet ca392ffec52242448c95ad9c371f0b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:26 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:51954
I20260808 00:37:26.882218 3776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:26.882359 3776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:26.883507 4265 tablet_service.cc:1467] Processing CreateTablet for tablet 20f74527aefa4c029d367f55a8926107 (DEFAULT_TABLE table=test-table [id=cdb9179e2973474982db72f750e583a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.883613 4265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f74527aefa4c029d367f55a8926107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.884784 4266 tablet_service.cc:1514] Processing DeleteTablet for tablet ca392ffec52242448c95ad9c371f0b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:26 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:57524
I20260808 00:37:26.885022 4711 tablet_replica.cc:330] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: stopping tablet replica
I20260808 00:37:26.885113 4711 raft_consensus.cc:2229] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:26.885214 4711 raft_consensus.cc:2258] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:26.885658 4676 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Bootstrap starting.
I20260808 00:37:26.885732 4711 ts_tablet_manager.cc:1905] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:26.885802 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 20f74527aefa4c029d367f55a8926107 (DEFAULT_TABLE table=test-table [id=cdb9179e2973474982db72f750e583a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.885902 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f74527aefa4c029d367f55a8926107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.886693 4676 tablet_bootstrap.cc:654] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.887696 4681 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Bootstrap starting.
I20260808 00:37:26.887914 4711 ts_tablet_manager.cc:1918] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:26.887997 4711 log.cc:1196] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/ca392ffec52242448c95ad9c371f0b18
I20260808 00:37:26.888525 4681 tablet_bootstrap.cc:654] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.888695 4711 ts_tablet_manager.cc:1939] T ca392ffec52242448c95ad9c371f0b18 P 11028bd5d2fe4e1d9289713278212b2f: Deleting consensus metadata
I20260808 00:37:26.889128 3757 catalog_manager.cc:4841] TS 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077): tablet ca392ffec52242448c95ad9c371f0b18 (table test-table [id=260e227c0fe24f2286c3aef6ba7e5598]) successfully deleted
I20260808 00:37:26.889595 4515 tablet_service.cc:1467] Processing CreateTablet for tablet 20f74527aefa4c029d367f55a8926107 (DEFAULT_TABLE table=test-table [id=cdb9179e2973474982db72f750e583a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:26.889707 4515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 20f74527aefa4c029d367f55a8926107. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.891238 4677 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Bootstrap starting.
I20260808 00:37:26.891579 4676 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: No bootstrap required, opened a new log
I20260808 00:37:26.891630 4676 ts_tablet_manager.cc:1397] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260808 00:37:26.891731 4709 tablet_replica.cc:330] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: stopping tablet replica
I20260808 00:37:26.891793 4676 raft_consensus.cc:348] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.891839 4709 raft_consensus.cc:2229] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:26.891868 4676 raft_consensus.cc:374] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.891891 4676 raft_consensus.cc:729] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11028bd5d2fe4e1d9289713278212b2f, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.891939 4709 raft_consensus.cc:2258] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:26.891937 4676 consensus_queue.cc:260] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.892036 4676 ts_tablet_manager.cc:1428] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:26.892169 4681 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: No bootstrap required, opened a new log
I20260808 00:37:26.892211 4681 ts_tablet_manager.cc:1397] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260808 00:37:26.892314 4677 tablet_bootstrap.cc:654] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.892669 4709 ts_tablet_manager.cc:1905] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:26.892695 4681 raft_consensus.cc:348] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.892774 4681 raft_consensus.cc:374] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.892796 4681 raft_consensus.cc:729] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07498aba7d3b426487a8d96c3af5fb29, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.892846 4681 consensus_queue.cc:260] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.893082 4681 ts_tablet_manager.cc:1428] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:26.893749 4677 tablet_bootstrap.cc:492] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: No bootstrap required, opened a new log
I20260808 00:37:26.893793 4677 ts_tablet_manager.cc:1397] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:37:26.894138 4677 raft_consensus.cc:348] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.894200 4677 raft_consensus.cc:374] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.894218 4677 raft_consensus.cc:729] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d327b67104cb467fae3f3469723d9b85, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.894256 4677 consensus_queue.cc:260] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.894340 4677 ts_tablet_manager.cc:1428] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:26.894912 4709 ts_tablet_manager.cc:1918] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:26.894995 4709 log.cc:1196] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/ca392ffec52242448c95ad9c371f0b18
I20260808 00:37:26.895494 4709 ts_tablet_manager.cc:1939] T ca392ffec52242448c95ad9c371f0b18 P 07498aba7d3b426487a8d96c3af5fb29: Deleting consensus metadata
I20260808 00:37:26.895881 3754 catalog_manager.cc:4841] TS 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043): tablet ca392ffec52242448c95ad9c371f0b18 (table test-table [id=260e227c0fe24f2286c3aef6ba7e5598]) successfully deleted
I20260808 00:37:26.897578 4710 tablet_replica.cc:330] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: stopping tablet replica
I20260808 00:37:26.897696 4710 raft_consensus.cc:2229] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:26.897812 4710 raft_consensus.cc:2258] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:26.898319 4710 ts_tablet_manager.cc:1905] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:26.901008 4710 ts_tablet_manager.cc:1918] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:26.901082 4710 log.cc:1196] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/ca392ffec52242448c95ad9c371f0b18
I20260808 00:37:26.901367 4710 ts_tablet_manager.cc:1939] T ca392ffec52242448c95ad9c371f0b18 P d327b67104cb467fae3f3469723d9b85: Deleting consensus metadata
I20260808 00:37:26.901770 3757 catalog_manager.cc:4841] TS d327b67104cb467fae3f3469723d9b85 (127.1.208.3:37839): tablet ca392ffec52242448c95ad9c371f0b18 (table test-table [id=260e227c0fe24f2286c3aef6ba7e5598]) successfully deleted
I20260808 00:37:26.946215 4705 raft_consensus.cc:482] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:26.946302 4705 raft_consensus.cc:504] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.946597 4705 leader_election.cc:288] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043), 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077)
I20260808 00:37:26.946753 4053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f74527aefa4c029d367f55a8926107" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07498aba7d3b426487a8d96c3af5fb29" is_pre_election: true
I20260808 00:37:26.946785 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f74527aefa4c029d367f55a8926107" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f" is_pre_election: true
I20260808 00:37:26.946868 4295 raft_consensus.cc:2454] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 0.
I20260808 00:37:26.946869 4053 raft_consensus.cc:2454] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 0.
I20260808 00:37:26.947022 4426 leader_election.cc:302] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07498aba7d3b426487a8d96c3af5fb29, d327b67104cb467fae3f3469723d9b85; no voters:
I20260808 00:37:26.947115 4705 raft_consensus.cc:2781] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:26.947147 4705 raft_consensus.cc:482] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:26.947165 4705 raft_consensus.cc:3037] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.948639 4705 raft_consensus.cc:504] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.948765 4705 leader_election.cc:288] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 election: Requested vote from peers 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043), 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077)
I20260808 00:37:26.948945 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f74527aefa4c029d367f55a8926107" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f"
I20260808 00:37:26.949021 4295 raft_consensus.cc:3037] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.948966 4053 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "20f74527aefa4c029d367f55a8926107" candidate_uuid: "d327b67104cb467fae3f3469723d9b85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07498aba7d3b426487a8d96c3af5fb29"
I20260808 00:37:26.949064 4053 raft_consensus.cc:3037] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:26.949952 4053 raft_consensus.cc:2454] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 1.
I20260808 00:37:26.950150 4426 leader_election.cc:302] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07498aba7d3b426487a8d96c3af5fb29, d327b67104cb467fae3f3469723d9b85; no voters:
I20260808 00:37:26.950222 4295 raft_consensus.cc:2454] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d327b67104cb467fae3f3469723d9b85 in term 1.
I20260808 00:37:26.950284 4705 raft_consensus.cc:2781] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:26.950351 4705 raft_consensus.cc:686] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 1 LEADER]: Becoming Leader. State: Replica: d327b67104cb467fae3f3469723d9b85, State: Running, Role: LEADER
I20260808 00:37:26.950397 4705 consensus_queue.cc:237] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.950956 3778 catalog_manager.cc:5495] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 reported cstate change: term changed from 0 to 1, leader changed from <none> to d327b67104cb467fae3f3469723d9b85 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "d327b67104cb467fae3f3469723d9b85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } health_report { overall_health: HEALTHY } } }
Aug 08 00:37:26 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:26.980937 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.974136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33881 (local address 127.0.0.1:50500)
0808 00:37:26.974387 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:26.974398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:26.974463 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:26.974602 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:26.974603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:26.975046 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:26.975096 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.975099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.975721 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.975722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.976249 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:26.976250 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.976564 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.976565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:26.976613 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.977164 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:26.977168 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:26.978584 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:26.979983 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.979990 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.979993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.980169 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.980339 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:26.980340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:26.980340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:26.980393 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:26.980598 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:26.980606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:26.980652 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:37:26.980710 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:37:26.980829 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":130,"threads_started":1}
I20260808 00:37:26.981017 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:26.974163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.1.208.62:33881)
0808 00:37:26.974226 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:26.974228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:26.974459 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:37:26.974537 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:26.974538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:26.974550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:26.974598 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:26.975174 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.975654 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.976322 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:26.976507 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:26.976559 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:26.978659 (+ 2100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:26.978660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:26.978660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:26.978667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:26.979916 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.980229 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.980231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.980231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.980281 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:26.980441 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:26.980442 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:26.980442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:26.980546 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:26.980600 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:26.980736 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 00:37:26.980923 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":32}
I20260808 00:37:26.983319 3778 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:50500:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:37:26.983649 3778 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:26.985253 4515 tablet_service.cc:1467] Processing CreateTablet for tablet 0931359fd8e44e3eb4086efb52d8b261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b63cc3895f644acfa275ade4275bf61d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:26.985368 4013 tablet_service.cc:1467] Processing CreateTablet for tablet 0931359fd8e44e3eb4086efb52d8b261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b63cc3895f644acfa275ade4275bf61d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:26.985431 4265 tablet_service.cc:1467] Processing CreateTablet for tablet 0931359fd8e44e3eb4086efb52d8b261 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b63cc3895f644acfa275ade4275bf61d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:26.985509 4515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931359fd8e44e3eb4086efb52d8b261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.985538 4013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931359fd8e44e3eb4086efb52d8b261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.985558 4265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0931359fd8e44e3eb4086efb52d8b261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:26.987532 4681 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: Bootstrap starting.
I20260808 00:37:26.987533 4676 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: Bootstrap starting.
I20260808 00:37:26.988714 4681 tablet_bootstrap.cc:654] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.988758 4676 tablet_bootstrap.cc:654] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.989814 4677 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: Bootstrap starting.
I20260808 00:37:26.990283 4676 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: No bootstrap required, opened a new log
I20260808 00:37:26.990337 4676 ts_tablet_manager.cc:1397] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:37:26.990612 4677 tablet_bootstrap.cc:654] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:26.990754 4676 raft_consensus.cc:348] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.990840 4676 raft_consensus.cc:374] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.990859 4676 raft_consensus.cc:729] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11028bd5d2fe4e1d9289713278212b2f, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.990979 4676 consensus_queue.cc:260] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.991246 4682 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:26.991422 4682 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:26.991753 4676 ts_tablet_manager.cc:1428] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:26.992084 4677 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: No bootstrap required, opened a new log
I20260808 00:37:26.992133 4677 ts_tablet_manager.cc:1397] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:26.992568 4677 raft_consensus.cc:348] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.992635 4677 raft_consensus.cc:374] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.992647 4677 raft_consensus.cc:729] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d327b67104cb467fae3f3469723d9b85, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.992681 4677 consensus_queue.cc:260] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.992762 4677 ts_tablet_manager.cc:1428] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:26.992776 4705 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:26.992852 4705 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:26.993371 4681 tablet_bootstrap.cc:492] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: No bootstrap required, opened a new log
I20260808 00:37:26.993415 4681 ts_tablet_manager.cc:1397] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260808 00:37:26.993752 4681 raft_consensus.cc:348] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.993813 4681 raft_consensus.cc:374] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:26.993826 4681 raft_consensus.cc:729] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07498aba7d3b426487a8d96c3af5fb29, State: Initialized, Role: FOLLOWER
I20260808 00:37:26.993860 4681 consensus_queue.cc:260] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:26.993952 4681 ts_tablet_manager.cc:1428] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:26.994064 4686 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:26.994140 4686 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:27.105474 4686 raft_consensus.cc:482] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:27.105576 4686 raft_consensus.cc:504] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:27.105939 4686 leader_election.cc:288] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077), d327b67104cb467fae3f3469723d9b85 (127.1.208.3:37839)
I20260808 00:37:27.109640 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.106316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:48579 (local address 127.1.208.2:36077)
0808 00:37:27.106383 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.106385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.106437 (+ 52us) server_negotiation.cc:407] Connection header received
0808 00:37:27.106519 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.106520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.106531 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.106579 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:27.106862 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.107368 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.108683 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.109122 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.109186 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.109199 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.109533 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.109579 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:37:27.109642 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.106227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36077 (local address 127.1.208.1:48579)
0808 00:37:27.106359 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:27.106372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:27.106458 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:27.106605 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:27.106607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:27.106609 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:27.106770 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.106772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.107444 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.107445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.108593 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.108596 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.109349 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.109351 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.109455 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.109460 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:27.109512 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:37:27.109554 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260808 00:37:27.109697 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.106492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:46997 (local address 127.1.208.3:37839)
0808 00:37:27.106566 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.106568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.106722 (+ 154us) server_negotiation.cc:407] Connection header received
0808 00:37:27.106772 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.106773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.106781 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.106829 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:27.107162 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.107626 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.108743 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.109145 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.109207 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.109218 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.109591 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.109634 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 00:37:27.109869 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931359fd8e44e3eb4086efb52d8b261" candidate_uuid: "07498aba7d3b426487a8d96c3af5fb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f" is_pre_election: true
I20260808 00:37:27.109932 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.106401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:37839 (local address 127.1.208.1:46997)
0808 00:37:27.106659 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:27.106667 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:27.106718 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:27.106861 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:27.106862 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:27.106863 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:27.107094 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.107097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.107692 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.107694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.108656 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.108659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.109362 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.109364 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.109518 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.109522 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:27.109576 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:37:27.109616 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":137,"threads_started":1}
I20260808 00:37:27.109959 4295 raft_consensus.cc:2454] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07498aba7d3b426487a8d96c3af5fb29 in term 0.
I20260808 00:37:27.110175 3942 leader_election.cc:302] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07498aba7d3b426487a8d96c3af5fb29, 11028bd5d2fe4e1d9289713278212b2f; no voters:
I20260808 00:37:27.110150 4559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931359fd8e44e3eb4086efb52d8b261" candidate_uuid: "07498aba7d3b426487a8d96c3af5fb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d327b67104cb467fae3f3469723d9b85" is_pre_election: true
I20260808 00:37:27.110262 4559 raft_consensus.cc:2454] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07498aba7d3b426487a8d96c3af5fb29 in term 0.
I20260808 00:37:27.110320 4686 raft_consensus.cc:2781] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:27.110365 4686 raft_consensus.cc:482] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:27.110381 4686 raft_consensus.cc:3037] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:27.111595 4686 raft_consensus.cc:504] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:27.111703 4686 leader_election.cc:288] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [CANDIDATE]: Term 1 election: Requested vote from peers 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077), d327b67104cb467fae3f3469723d9b85 (127.1.208.3:37839)
I20260808 00:37:27.111852 4295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931359fd8e44e3eb4086efb52d8b261" candidate_uuid: "07498aba7d3b426487a8d96c3af5fb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11028bd5d2fe4e1d9289713278212b2f"
I20260808 00:37:27.111872 4559 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0931359fd8e44e3eb4086efb52d8b261" candidate_uuid: "07498aba7d3b426487a8d96c3af5fb29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d327b67104cb467fae3f3469723d9b85"
I20260808 00:37:27.111920 4295 raft_consensus.cc:3037] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:27.111945 4559 raft_consensus.cc:3037] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:27.112775 4295 raft_consensus.cc:2454] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07498aba7d3b426487a8d96c3af5fb29 in term 1.
I20260808 00:37:27.112834 4559 raft_consensus.cc:2454] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07498aba7d3b426487a8d96c3af5fb29 in term 1.
I20260808 00:37:27.113013 3942 leader_election.cc:302] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07498aba7d3b426487a8d96c3af5fb29, d327b67104cb467fae3f3469723d9b85; no voters:
I20260808 00:37:27.113101 4686 raft_consensus.cc:2781] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:27.113255 4686 raft_consensus.cc:686] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 LEADER]: Becoming Leader. State: Replica: 07498aba7d3b426487a8d96c3af5fb29, State: Running, Role: LEADER
I20260808 00:37:27.113389 4686 consensus_queue.cc:237] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } }
I20260808 00:37:27.113787 4743 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: TxnStatusTablet state changed. Reason: New leader 07498aba7d3b426487a8d96c3af5fb29. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.113881 4743 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:27.114111 3778 catalog_manager.cc:5495] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1). New cstate: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:27.114225 4745 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:37:27.114336 4745 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:37:27.139983 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.137384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.1.208.1:37043)
0808 00:37:27.137439 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.137441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.137448 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:27.137480 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.137481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.137497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.137554 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:27.137714 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.138181 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.139253 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.139442 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.139488 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.139681 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.139785 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.139842 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.139903 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.139920 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:37:27.140043 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.137134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37043 (local address 127.0.0.1:51962)
0808 00:37:27.137227 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:27.137242 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:27.137323 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:27.137584 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:27.137585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:27.137587 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:27.137645 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.137648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.138251 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.138252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.139177 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.139181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.139495 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.139497 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.139574 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.139594 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.139836 (+ 242us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.139843 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:37:27.139898 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:37:27.139954 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:37:27.141399 4295 raft_consensus.cc:1261] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Refusing update from remote peer 07498aba7d3b426487a8d96c3af5fb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:27.141522 4559 raft_consensus.cc:1261] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Refusing update from remote peer 07498aba7d3b426487a8d96c3af5fb29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:27.141675 4743 consensus_queue.cc:1035] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:27.141819 4686 consensus_queue.cc:1035] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:27.142594 4705 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: TxnStatusTablet state changed. Reason: New leader 07498aba7d3b426487a8d96c3af5fb29. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.142606 4682 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: TxnStatusTablet state changed. Reason: New leader 07498aba7d3b426487a8d96c3af5fb29. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.142688 4705 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:27.142702 4682 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:27.143594 4686 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.143690 4686 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:27.143743 4686 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.143718 4705 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.143792 4686 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 07498aba7d3b426487a8d96c3af5fb29: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:27.143797 4705 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P d327b67104cb467fae3f3469723d9b85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:27.143966 4695 mvcc.cc:205] Tried to move back new op lower bound from 7316068135488815104 to 7316068135376896000. Current Snapshot: MvccSnapshot[applied={T|T < 7316068135488815104}]
I20260808 00:37:27.145087 4682 tablet_replica.cc:439] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "07498aba7d3b426487a8d96c3af5fb29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 } } peers { permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 } } peers { permanent_uuid: "d327b67104cb467fae3f3469723d9b85" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 37839 } } }
I20260808 00:37:27.145186 4682 tablet_replica.cc:442] T 0931359fd8e44e3eb4086efb52d8b261 P 11028bd5d2fe4e1d9289713278212b2f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:27.150557 4760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.147781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37043 (local address 127.1.208.3:44509)
0808 00:37:27.147891 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:27.147902 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:27.147955 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:27.148083 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:27.148084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:27.148085 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:27.148127 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.148129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.149009 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.149010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.149687 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.149689 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.150136 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.150137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.150178 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.150193 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.150377 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.150379 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:27.150449 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:37:27.150500 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"thread_start_us":51,"threads_started":1}
I20260808 00:37:27.150556 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.147840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:44509 (local address 127.1.208.1:37043)
0808 00:37:27.147902 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.147904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.147948 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:37:27.148016 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.148017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.148027 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.148081 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:27.148180 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.148941 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.149758 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.150066 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.150146 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.150264 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.150321 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.150371 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.150461 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.150482 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:37:27.153321 4053 raft_consensus.cc:1261] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Refusing update from remote peer d327b67104cb467fae3f3469723d9b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:27.153367 4295 raft_consensus.cc:1261] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Refusing update from remote peer d327b67104cb467fae3f3469723d9b85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:27.153530 4705 consensus_queue.cc:1035] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07498aba7d3b426487a8d96c3af5fb29" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:27.153651 4684 consensus_queue.cc:1035] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "11028bd5d2fe4e1d9289713278212b2f" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:27.156450 4696 mvcc.cc:205] Tried to move back new op lower bound from 7316068135538368512 to 7316068134709161984. Current Snapshot: MvccSnapshot[applied={T|T < 7316068135538368512}]
I20260808 00:37:27.165081 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.161854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:50939 (local address 127.1.208.3:37839)
0808 00:37:27.161897 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.161899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.162346 (+ 447us) server_negotiation.cc:407] Connection header received
0808 00:37:27.162395 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.162396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.162410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.162456 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:27.162741 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.163437 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.164382 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.164581 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.164634 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.164842 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.164878 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.164922 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.164979 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.165001 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:27.165571 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.161838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:37839 (local address 127.1.208.1:50939)
0808 00:37:27.162264 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:27.162276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:27.162345 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:27.162457 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:27.162458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:27.162459 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:27.162661 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.162664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.163502 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.163504 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.164303 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:27.164306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.164639 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.164641 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:27.164744 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.164762 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:27.164928 (+ 166us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:27.164930 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:27.164972 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 00:37:27.165020 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":208,"threads_started":1}
I20260808 00:37:27.173949 3776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:27.174067 3776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:27.174738 3776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2072b6dd1c914cc493c5edfd320a5a12: Sending DeleteTablet for 3 replicas of tablet 20f74527aefa4c029d367f55a8926107
I20260808 00:37:27.175009 4265 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f74527aefa4c029d367f55a8926107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:57524
I20260808 00:37:27.175024 4515 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f74527aefa4c029d367f55a8926107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:48010
I20260808 00:37:27.175117 4711 tablet_replica.cc:330] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: stopping tablet replica
I20260808 00:37:27.175133 4710 tablet_replica.cc:330] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: stopping tablet replica
I20260808 00:37:27.175169 4711 raft_consensus.cc:2229] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:27.175185 4710 raft_consensus.cc:2229] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:27.175252 4710 raft_consensus.cc:2258] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:27.175271 4711 raft_consensus.cc:2258] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:27.175591 4711 ts_tablet_manager.cc:1905] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:27.175594 4710 ts_tablet_manager.cc:1905] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:27.177235 4711 ts_tablet_manager.cc:1918] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:27.177330 4711 log.cc:1196] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/20f74527aefa4c029d367f55a8926107
I20260808 00:37:27.177363 4013 tablet_service.cc:1514] Processing DeleteTablet for tablet 20f74527aefa4c029d367f55a8926107 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:51954
I20260808 00:37:27.177484 4709 tablet_replica.cc:330] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: stopping tablet replica
I20260808 00:37:27.177537 4711 ts_tablet_manager.cc:1939] T 20f74527aefa4c029d367f55a8926107 P 11028bd5d2fe4e1d9289713278212b2f: Deleting consensus metadata
I20260808 00:37:27.177554 4709 raft_consensus.cc:2229] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:27.177594 4709 raft_consensus.cc:2258] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:27.177898 3757 catalog_manager.cc:4841] TS 11028bd5d2fe4e1d9289713278212b2f (127.1.208.2:36077): tablet 20f74527aefa4c029d367f55a8926107 (table test-table [id=cdb9179e2973474982db72f750e583a5]) successfully deleted
I20260808 00:37:27.178328 4710 ts_tablet_manager.cc:1918] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:27.178393 4710 log.cc:1196] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/20f74527aefa4c029d367f55a8926107
I20260808 00:37:27.178591 4710 ts_tablet_manager.cc:1939] T 20f74527aefa4c029d367f55a8926107 P d327b67104cb467fae3f3469723d9b85: Deleting consensus metadata
I20260808 00:37:27.178978 3757 catalog_manager.cc:4841] TS d327b67104cb467fae3f3469723d9b85 (127.1.208.3:37839): tablet 20f74527aefa4c029d367f55a8926107 (table test-table [id=cdb9179e2973474982db72f750e583a5]) successfully deleted
I20260808 00:37:27.179139 4709 ts_tablet_manager.cc:1905] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:27.180830 4709 ts_tablet_manager.cc:1918] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:27.180898 4709 log.cc:1196] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/20f74527aefa4c029d367f55a8926107
I20260808 00:37:27.181093 4709 ts_tablet_manager.cc:1939] T 20f74527aefa4c029d367f55a8926107 P 07498aba7d3b426487a8d96c3af5fb29: Deleting consensus metadata
I20260808 00:37:27.181447 3754 catalog_manager.cc:4841] TS 07498aba7d3b426487a8d96c3af5fb29 (127.1.208.1:37043): tablet 20f74527aefa4c029d367f55a8926107 (table test-table [id=cdb9179e2973474982db72f750e583a5]) successfully deleted
Aug 08 00:37:27 dist-test-slave-gjw7 krb5kdc[3711](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149446, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:27.188083 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.178344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.1.208.1:37043)
0808 00:37:27.178391 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.178393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.178647 (+ 254us) server_negotiation.cc:407] Connection header received
0808 00:37:27.178774 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.178775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.178786 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.178833 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:27.179663 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.180147 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.181025 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.181280 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.181363 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.184716 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:27.184731 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:27.184732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:27.184741 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:27.186677 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:27.187050 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:27.187054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:27.187055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:27.187128 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:27.187284 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:27.187284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:27.187285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:27.187482 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:27.187780 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.187825 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.187914 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:37:27.188609 4131 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51964
I20260808 00:37:27.195992 3903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:27.189760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.1.208.62:33881)
0808 00:37:27.189818 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:27.189821 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:27.189830 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:27.189896 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:27.189897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:27.189909 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:27.189948 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:27.190558 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.191061 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.191730 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:27.191923 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:27.191976 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:27.193523 (+ 1547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:27.193525 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:27.193525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:27.193533 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:27.194901 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:27.195224 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:27.195225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:27.195226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:27.195284 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:27.195487 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:27.195487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:27.195488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:27.195603 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:27.195695 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:27.195791 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:37:27.195885 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260808 00:37:27.196240 3776 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50502
I20260808 00:37:27.197052 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3911
I20260808 00:37:27.214036 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4152
I20260808 00:37:27.233968 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4399
I20260808 00:37:27.261161 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3736
2026-08-08T00:37:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2648 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:27 dist-test-slave-gjw7 krb5kdc[4781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:27 dist-test-slave-gjw7 krb5kdc[4781](info): set up 2 sockets
Aug 08 00:37:27 dist-test-slave-gjw7 krb5kdc[4781](info): commencing operation
krb5kdc: starting...
W20260808 00:37:28.434093 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:28.558081 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37203
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:28.640285 4804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:28.640489 4804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:28.640511 4804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:28.641649 4804 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:28.641686 4804 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:37:28.641698 4804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:28.641710 4804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:28.641721 4804 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:28.641732 4804 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:28.643148 4804 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37203
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:28.643348 4804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:28.643515 4804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:28.648159 4817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:28.648159 4814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:28.648177 4813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:28.648532 4804 server_base.cc:1034] running on GCE node
I20260808 00:37:28.648809 4804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:28.649096 4804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:28.650240 4804 hybrid_clock.cc:648] HybridClock initialized: now 1786149448650226 us; error 27 us; skew 500 ppm
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:28.652823 4804 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:28.653321 4804 webserver.cc:460] Webserver started at http://127.1.208.62:39773/ using document root <none> and password file <none>
I20260808 00:37:28.653522 4804 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:28.653564 4804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:28.653653 4804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:28.654726 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "e669983ed7874e5b9ae1d88a1dc23753"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "7f97f33faff9b406b6bfef12578d56db"
server_key_iv: "dc364ecc196499b20c9481d399e85fa6"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.655144 4804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "e669983ed7874e5b9ae1d88a1dc23753"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "7f97f33faff9b406b6bfef12578d56db"
server_key_iv: "dc364ecc196499b20c9481d399e85fa6"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.656533 4804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:28.657204 4826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:28.657387 4804 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:28.657469 4804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "e669983ed7874e5b9ae1d88a1dc23753"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "7f97f33faff9b406b6bfef12578d56db"
server_key_iv: "dc364ecc196499b20c9481d399e85fa6"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.657522 4804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:28.665222 4804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:28.667075 4804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:28.667181 4804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:28.671746 4804 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37203
I20260808 00:37:28.671831 4952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37203 every 8 connection(s)
I20260808 00:37:28.672209 4804 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:28.672927 4953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:28.673184 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4804
I20260808 00:37:28.673285 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:28.673506 1856 external_mini_cluster.cc:1428] Setting key 55bdd91585d39e2c9c95c5387da77cf1
I20260808 00:37:28.675957 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Bootstrap starting.
I20260808 00:37:28.676697 4953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:28.676998 4953 log.cc:824] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:28.677827 4953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: No bootstrap required, opened a new log
I20260808 00:37:28.680123 4953 raft_consensus.cc:348] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } }
I20260808 00:37:28.680306 4953 raft_consensus.cc:374] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:28.680341 4953 raft_consensus.cc:729] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e669983ed7874e5b9ae1d88a1dc23753, State: Initialized, Role: FOLLOWER
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:28.680472 4953 consensus_queue.cc:260] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } }
I20260808 00:37:28.680559 4953 raft_consensus.cc:388] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:28.680588 4953 raft_consensus.cc:482] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:28.680636 4953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:28.681272 4953 raft_consensus.cc:504] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } }
I20260808 00:37:28.681402 4953 leader_election.cc:302] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e669983ed7874e5b9ae1d88a1dc23753; no voters:
I20260808 00:37:28.681573 4953 leader_election.cc:288] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:28.681617 4959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:28.681770 4959 raft_consensus.cc:686] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [term 1 LEADER]: Becoming Leader. State: Replica: e669983ed7874e5b9ae1d88a1dc23753, State: Running, Role: LEADER
I20260808 00:37:28.681843 4953 sys_catalog.cc:564] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:28.681878 4959 consensus_queue.cc:237] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } }
I20260808 00:37:28.682267 4962 sys_catalog.cc:455] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e669983ed7874e5b9ae1d88a1dc23753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } } }
I20260808 00:37:28.682375 4962 sys_catalog.cc:458] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:28.683455 4960 sys_catalog.cc:455] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e669983ed7874e5b9ae1d88a1dc23753. Latest consensus state: current_term: 1 leader_uuid: "e669983ed7874e5b9ae1d88a1dc23753" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e669983ed7874e5b9ae1d88a1dc23753" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37203 } } }
I20260808 00:37:28.683545 4960 sys_catalog.cc:458] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753 [sys.catalog]: This master's current role is: LEADER
W20260808 00:37:28.683718 4987 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:28.683800 4987 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:28.684237 4977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:28.684417 4977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:28.685141 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:28.674094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40192 (local address 127.1.208.62:37203)
0808 00:37:28.674271 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:28.674282 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:28.674314 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:37:28.674832 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:28.674842 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:28.674907 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:28.675054 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:28.675685 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.676982 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.677754 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.678329 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.678454 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:28.680971 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:28.680975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:28.680975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:28.680987 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:28.683146 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.683575 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.683579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.683579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.683684 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.683895 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.683896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.683897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.684204 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:28.684333 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:28.684802 (+ 469us) server_negotiation.cc:299] Negotiation successful
0808 00:37:28.684942 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260808 00:37:28.686022 4977 catalog_manager.cc:1347] Generated new cluster ID: cb859d8aa07b478786f2647562ec47e5
I20260808 00:37:28.686076 4977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:28.694882 4977 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:28.695441 4977 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:28.702965 4977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Generated new TSK 0
I20260808 00:37:28.703167 4977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:28.731617 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37203
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:28.813105 4996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:28.813337 4996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:28.813364 4996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:28.814546 4996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:28.814587 4996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:28.814606 4996 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:37:28.814617 4996 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:28.814635 4996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:28.816265 4996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:28.816493 4996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:28.816722 4996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:37:28.821532 4996 server_base.cc:1034] running on GCE node
W20260808 00:37:28.821499 5008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:28.821485 5009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:28.821487 5013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:28.821931 4996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:28.822176 4996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:28.823315 4996 hybrid_clock.cc:648] HybridClock initialized: now 1786149448823300 us; error 30 us; skew 500 ppm
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:28.825863 4996 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:28.826329 4996 webserver.cc:460] Webserver started at http://127.1.208.1:42847/ using document root <none> and password file <none>
I20260808 00:37:28.826552 4996 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:28.826601 4996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:28.826752 4996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:28.827818 4996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b3746d617d8c969ccfa991bdddc1e114"
server_key_iv: "450da790be00bf179935509363c5566d"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.828161 4996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b3746d617d8c969ccfa991bdddc1e114"
server_key_iv: "450da790be00bf179935509363c5566d"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.829579 4996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:28.830371 5022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:28.830549 4996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:28.830638 4996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b3746d617d8c969ccfa991bdddc1e114"
server_key_iv: "450da790be00bf179935509363c5566d"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.830699 4996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:28.855877 4996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:28.858166 4996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:28.858309 4996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:28.858879 4996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:28.858937 4996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:28.858973 4996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:28.858999 4996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:28.865773 4996 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:37067
I20260808 00:37:28.865902 5205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:37067 every 8 connection(s)
I20260808 00:37:28.866180 4996 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:28.866761 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4996
I20260808 00:37:28.866858 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:28.867034 1856 external_mini_cluster.cc:1428] Setting key 995e474b57a6bcb6e583bb97f7ebcb3e
I20260808 00:37:28.869504 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:28.859527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:41423 (local address 127.1.208.62:37203)
0808 00:37:28.859579 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:28.859581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:28.859834 (+ 253us) server_negotiation.cc:407] Connection header received
0808 00:37:28.860437 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:28.860438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:28.860452 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:28.860505 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:28.861511 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.862296 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.863717 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.863994 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.864063 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:28.866693 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:28.866695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:28.866695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:28.866712 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:28.868101 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.868541 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.868541 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.868542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.868590 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.868829 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.868830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.868830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.868934 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:28.868979 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:28.869316 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 00:37:28.869407 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:37:28.869719 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:28.859537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.1:41423)
0808 00:37:28.859708 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:28.859747 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:28.860324 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:28.860676 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:28.860681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:28.861085 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:28.861392 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:28.861401 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.862381 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.862382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:28.863632 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:28.863636 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.864088 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.864089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:28.864269 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:28.864804 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:28.864816 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:28.866610 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:28.868182 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:28.868190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:28.868192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:28.868474 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:28.868641 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:28.868642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:28.868642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:28.868780 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:28.868984 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:28.868990 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:28.869166 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 00:37:28.869273 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:37:28.869475 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260808 00:37:28.870304 5206 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37203
I20260808 00:37:28.870389 5206 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:28.870569 5206 heartbeater.cc:511] Master 127.1.208.62:37203 requested a full tablet report, sending...
I20260808 00:37:28.871142 4861 ts_manager.cc:194] Registered new tserver with Master: 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067)
I20260808 00:37:28.871982 4861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:41423
I20260808 00:37:28.877086 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:28.871408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:55131 (local address 127.1.208.62:37203)
0808 00:37:28.871452 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:28.871454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:28.871633 (+ 179us) server_negotiation.cc:407] Connection header received
0808 00:37:28.871688 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:28.871688 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:28.871698 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:28.871748 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:28.872037 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.872731 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.873744 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.873925 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.873973 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:28.874913 (+ 940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:28.874914 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:28.874914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:28.874920 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:28.876108 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.876468 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.876469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.876469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.876510 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:28.876681 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:28.876682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:28.876682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:28.876766 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:28.876808 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:28.876961 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 00:37:28.877026 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:37:28.877139 5216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:28.871411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.1:55131)
0808 00:37:28.871560 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:28.871570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:28.871632 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:28.871752 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:28.871753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:28.871918 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:28.871973 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:28.871974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.872849 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.872851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:28.873651 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:28.873654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:28.873992 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:28.873993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:28.874073 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:28.874471 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:28.874476 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:28.874831 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:28.876184 (+ 1353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:28.876185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:28.876186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:28.876389 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:28.876558 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:28.876559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:28.876559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:28.876626 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:28.876817 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:28.876818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:28.876868 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 00:37:28.876943 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:37:28.877043 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:28.896391 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37203
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:28.979157 5218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:28.979363 5218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:28.979386 5218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:28.980501 5218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:28.980542 5218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:28.980561 5218 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:37:28.980572 5218 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:28.980598 5218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:28.982091 5218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:28.982307 5218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:28.982535 5218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:28.987528 5226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:28.987504 5227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:28.987660 5218 server_base.cc:1034] running on GCE node
W20260808 00:37:28.987506 5231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:28.987885 5218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:28.988147 5218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:28.989315 5218 hybrid_clock.cc:648] HybridClock initialized: now 1786149448989274 us; error 31 us; skew 500 ppm
Aug 08 00:37:28 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:28.991807 5218 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:28.992229 5218 webserver.cc:460] Webserver started at http://127.1.208.2:44521/ using document root <none> and password file <none>
I20260808 00:37:28.992419 5218 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:28.992462 5218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:28.992559 5218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:28.993744 5218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "42fe980ab4774997b4a67e6c95b3927e"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b7bafef7190ce39ad3ac509564548a3c"
server_key_iv: "ff94f65fe351291bf28d5828cd423dc7"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.994104 5218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "42fe980ab4774997b4a67e6c95b3927e"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b7bafef7190ce39ad3ac509564548a3c"
server_key_iv: "ff94f65fe351291bf28d5828cd423dc7"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.995453 5218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:28.996179 5242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:28.996338 5218 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:28.996423 5218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "42fe980ab4774997b4a67e6c95b3927e"
format_stamp: "Formatted at 2026-08-08 00:37:28 on dist-test-slave-gjw7"
server_key: "b7bafef7190ce39ad3ac509564548a3c"
server_key_iv: "ff94f65fe351291bf28d5828cd423dc7"
server_key_version: "encryptionkey@0"
I20260808 00:37:28.996481 5218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:29.010927 5218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:29.013360 5218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:29.013484 5218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:29.014007 5218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:29.014052 5218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:29.014076 5218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:29.014088 5218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:29.021119 5218 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:35427
I20260808 00:37:29.021200 5447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:35427 every 8 connection(s)
I20260808 00:37:29.021595 5218 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:29.023792 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.014563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:48423 (local address 127.1.208.62:37203)
0808 00:37:29.014609 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.014611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.014843 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:37:29.015349 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.015351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.015365 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.015428 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.016277 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.017034 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.018141 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.018447 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.018519 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.021317 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.021319 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.021319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.021328 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.022695 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.023083 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.023084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.023084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.023128 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.023305 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.023306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.023306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.023400 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.023443 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.023653 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.023728 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:29.023972 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.014538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.2:48423)
0808 00:37:29.014706 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.014744 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.015254 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.015509 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.015514 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.015859 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:29.016165 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.016173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.017134 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.017135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.018034 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.018037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.018552 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.018553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.018737 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.019324 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:29.019336 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:29.021207 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:29.022789 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.022796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.022797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.023019 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.023171 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.023171 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.023172 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.023263 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.023443 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:29.023446 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:29.023559 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.023638 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.023756 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260808 00:37:29.024416 5448 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37203
I20260808 00:37:29.024501 5448 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:29.024675 5448 heartbeater.cc:511] Master 127.1.208.62:37203 requested a full tablet report, sending...
I20260808 00:37:29.025034 4865 ts_manager.cc:194] Registered new tserver with Master: 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427)
I20260808 00:37:29.025509 4865 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:48423
I20260808 00:37:29.030239 5454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.025375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.2:56515)
0808 00:37:29.025504 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.025515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.025576 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.025705 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.025705 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.025903 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:29.025962 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.025963 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.026686 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.026687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.027189 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.027191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.027515 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.027516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.027569 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.027888 (+ 319us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:29.027892 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:29.028143 (+ 251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:29.029508 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.029508 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.029508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.029646 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.029775 (+ 129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.029775 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.029775 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.029815 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.030010 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:29.030011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:29.030059 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.030102 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.030175 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260808 00:37:29.030238 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.025442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:56515 (local address 127.1.208.62:37203)
0808 00:37:29.025489 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.025491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.025561 (+ 70us) server_negotiation.cc:407] Connection header received
0808 00:37:29.025641 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.025642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.025652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.025699 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.026031 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.026603 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.027291 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.027465 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.027510 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.028206 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.028214 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.028214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.028219 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.029453 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.029697 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.029698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.029698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.029741 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.029850 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.029851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.029851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.029972 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.030013 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.030122 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.030174 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:37:29.031602 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5218
I20260808 00:37:29.031688 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:29.031874 1856 external_mini_cluster.cc:1428] Setting key 9d90d4dd3326c9b0f9867abf4e7ea016
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:29.055958 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37203
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:33997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:29.137934 5459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:29.138130 5459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:29.138154 5459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:29.139237 5459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:29.139267 5459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:29.139285 5459 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 00:37:29.139302 5459 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:29.139317 5459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:29.140738 5459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:33997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:37203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5459
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:29.140939 5459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:29.141119 5459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:29.145885 5465 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:29.145901 5466 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:29.145956 5469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:29.146059 5459 server_base.cc:1034] running on GCE node
I20260808 00:37:29.146256 5459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:29.146479 5459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:29.147614 5459 hybrid_clock.cc:648] HybridClock initialized: now 1786149449147594 us; error 35 us; skew 500 ppm
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149449, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:29.149968 5459 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:29.150439 5459 webserver.cc:460] Webserver started at http://127.1.208.3:34063/ using document root <none> and password file <none>
I20260808 00:37:29.150641 5459 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:29.150687 5459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:29.150813 5459 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:29.151870 5459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "42e1b25066764f9a9c199bcdda877066"
format_stamp: "Formatted at 2026-08-08 00:37:29 on dist-test-slave-gjw7"
server_key: "8dd01f1660c3f6d4b5843325e8e28837"
server_key_iv: "e972a66bb0af45db97b34dcafce613b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:29.152212 5459 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "42e1b25066764f9a9c199bcdda877066"
format_stamp: "Formatted at 2026-08-08 00:37:29 on dist-test-slave-gjw7"
server_key: "8dd01f1660c3f6d4b5843325e8e28837"
server_key_iv: "e972a66bb0af45db97b34dcafce613b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:29.153517 5459 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:29.154198 5478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:29.154333 5459 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:29.154424 5459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "42e1b25066764f9a9c199bcdda877066"
format_stamp: "Formatted at 2026-08-08 00:37:29 on dist-test-slave-gjw7"
server_key: "8dd01f1660c3f6d4b5843325e8e28837"
server_key_iv: "e972a66bb0af45db97b34dcafce613b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:29.154486 5459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:29.171239 5459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:29.173000 5459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:29.173137 5459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:29.173851 5459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:29.173894 5459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:29.173925 5459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:29.173940 5459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149449, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:29.180747 5459 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:41829
I20260808 00:37:29.180795 5669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:41829 every 8 connection(s)
I20260808 00:37:29.181111 5459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:29.183365 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.174195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:34007 (local address 127.1.208.62:37203)
0808 00:37:29.174240 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.174242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.174466 (+ 224us) server_negotiation.cc:407] Connection header received
0808 00:37:29.174984 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.174985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.174999 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.175048 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.175699 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.176475 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.177990 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.178182 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.178240 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.180417 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.180418 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.180419 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.180426 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.181999 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.182380 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.182381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.182382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.182448 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.182676 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.182678 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.182678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.182795 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.182854 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.183160 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.183269 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:29.183504 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.174184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.3:34007)
0808 00:37:29.174330 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.174367 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.174876 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.175120 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.175124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.175419 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:29.175607 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.175612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.176581 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.176583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.177889 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.177893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.178278 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.178280 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.178394 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.178855 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:29.178864 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:29.180303 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:29.182077 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.182083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.182084 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.182305 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.182504 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.182505 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.182505 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.182605 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.182853 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:29.182860 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:29.183024 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.183131 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.183284 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260808 00:37:29.183916 5670 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37203
I20260808 00:37:29.184020 5670 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:29.184209 5670 heartbeater.cc:511] Master 127.1.208.62:37203 requested a full tablet report, sending...
I20260808 00:37:29.184597 4861 ts_manager.cc:194] Registered new tserver with Master: 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:29.185032 4861 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:34007
I20260808 00:37:29.189867 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.184984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:51177 (local address 127.1.208.62:37203)
0808 00:37:29.185023 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.185025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.185162 (+ 137us) server_negotiation.cc:407] Connection header received
0808 00:37:29.185217 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.185217 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.185226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.185268 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.185582 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.186035 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.186711 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.186892 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.186951 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.187933 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.187934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.187934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.187941 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.188881 (+ 940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.189175 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.189176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.189177 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.189227 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.189412 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.189413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.189414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.189522 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.189577 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.189711 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.189778 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:29.189881 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.184973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.1.208.3:51177)
0808 00:37:29.185105 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.185116 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.185175 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.185272 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.185273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.185459 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:29.185517 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.185519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.186122 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.186124 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.186642 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.186644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.186956 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.186957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.187034 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.187560 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:29.187564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:29.187848 (+ 284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:29.188960 (+ 1112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.188961 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.188961 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.189116 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.189279 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.189280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.189280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.189361 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.189576 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:29.189577 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:29.189621 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.189680 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.189779 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260808 00:37:29.191118 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5459
I20260808 00:37:29.191212 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:29.191416 1856 external_mini_cluster.cc:1428] Setting key a7fa353c4ae9dcfe9fae190fc2c8a21d
I20260808 00:37:29.192118 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:29.215487 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.209090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.1.208.62:37203)
0808 00:37:29.209148 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.209150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.209157 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:29.209180 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.209181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.209189 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.209233 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.209826 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.210326 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.211037 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.211227 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.211279 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.213321 (+ 2042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.213322 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.213323 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.213330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.214486 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.214786 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.214787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.214787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.214841 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.215001 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.215002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.215002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.215110 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.215197 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.215318 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.215394 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:29.217001 4861 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:29.218643 4861 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:29.225481 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.221018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43686 (local address 127.1.208.1:37067)
0808 00:37:29.221171 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.221177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.221692 (+ 515us) server_negotiation.cc:407] Connection header received
0808 00:37:29.221794 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.221797 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.221815 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.221899 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.222024 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.223012 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.224128 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.224978 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225047 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225102 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.225342 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.225382 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260808 00:37:29.225517 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.221074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.1.208.3:41829)
0808 00:37:29.221243 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.221247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.221458 (+ 211us) server_negotiation.cc:407] Connection header received
0808 00:37:29.221545 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.221547 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.221564 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.221641 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.221861 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.222960 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.224480 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.224973 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225037 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225101 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.225362 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.225416 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260808 00:37:29.225699 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.221152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.1.208.2:35427)
0808 00:37:29.221394 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.221397 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.221788 (+ 391us) server_negotiation.cc:407] Connection header received
0808 00:37:29.221831 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.221834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.221850 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.221912 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.222129 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.223148 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.224446 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.225146 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225209 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225263 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.225483 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.225598 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":170,"threads_started":1}
I20260808 00:37:29.225706 5698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.220805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35427 (local address 127.0.0.1:33988)
0808 00:37:29.221687 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.221695 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.221736 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.221970 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.221971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.221974 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.222042 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.222044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.223228 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.223229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.224364 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.224367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.225222 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.225337 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225343 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.225392 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.225649 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":865,"mutex_wait_us":3,"thread_start_us":111,"threads_started":1}
I20260808 00:37:29.225879 5700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.220803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41829 (local address 127.0.0.1:53880)
0808 00:37:29.221286 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.221388 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.221462 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.221729 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.221733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.221735 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.221802 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.221804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.223051 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.223052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.224373 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.224374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.225052 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225053 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.225130 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225144 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.225206 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.225259 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":53,"thread_start_us":51,"threads_started":1}
I20260808 00:37:29.226094 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.220803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.0.0.1:43686)
0808 00:37:29.221565 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.221574 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.221620 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.221916 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.221917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.221919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.221971 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.221972 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.223093 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.223094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.224052 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.224055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.225139 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.225141 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.225209 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.225213 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.225258 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.225587 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":749,"mutex_wait_us":5,"thread_start_us":94,"threads_started":1}
I20260808 00:37:29.226487 5325 tablet_service.cc:1467] Processing CreateTablet for tablet 282b781cfc70453ca7fc0c4b352588af (DEFAULT_TABLE table=test-table [id=6a90eb5138a2492e89dc7d0348092d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.226606 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 282b781cfc70453ca7fc0c4b352588af (DEFAULT_TABLE table=test-table [id=6a90eb5138a2492e89dc7d0348092d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.226820 5325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282b781cfc70453ca7fc0c4b352588af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.226739 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 282b781cfc70453ca7fc0c4b352588af (DEFAULT_TABLE table=test-table [id=6a90eb5138a2492e89dc7d0348092d26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.226961 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282b781cfc70453ca7fc0c4b352588af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.227032 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 282b781cfc70453ca7fc0c4b352588af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.229751 5706 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Bootstrap starting.
I20260808 00:37:29.230468 5706 tablet_bootstrap.cc:654] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.230577 5707 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Bootstrap starting.
I20260808 00:37:29.230792 5706 log.cc:824] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:29.231181 5708 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Bootstrap starting.
I20260808 00:37:29.231211 5707 tablet_bootstrap.cc:654] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.231559 5706 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: No bootstrap required, opened a new log
I20260808 00:37:29.231552 5707 log.cc:824] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:29.231623 5706 ts_tablet_manager.cc:1397] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:29.232087 5708 tablet_bootstrap.cc:654] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.232425 5708 log.cc:824] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:29.232532 5707 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: No bootstrap required, opened a new log
I20260808 00:37:29.232600 5707 ts_tablet_manager.cc:1397] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:29.233258 5708 tablet_bootstrap.cc:492] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: No bootstrap required, opened a new log
I20260808 00:37:29.233352 5708 ts_tablet_manager.cc:1397] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:29.233538 5706 raft_consensus.cc:348] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.233672 5706 raft_consensus.cc:374] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.233704 5706 raft_consensus.cc:729] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.233821 5706 consensus_queue.cc:260] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.234092 5706 ts_tablet_manager.cc:1428] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:37:29.234211 5448 heartbeater.cc:503] Master 127.1.208.62:37203 was elected leader, sending a full tablet report...
I20260808 00:37:29.234613 5707 raft_consensus.cc:348] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.234778 5707 raft_consensus.cc:374] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.234812 5707 raft_consensus.cc:729] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.234920 5707 consensus_queue.cc:260] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.235184 5707 ts_tablet_manager.cc:1428] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:37:29.235215 5206 heartbeater.cc:503] Master 127.1.208.62:37203 was elected leader, sending a full tablet report...
I20260808 00:37:29.235450 5708 raft_consensus.cc:348] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.235600 5708 raft_consensus.cc:374] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.235641 5708 raft_consensus.cc:729] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.235750 5708 consensus_queue.cc:260] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.236047 5708 ts_tablet_manager.cc:1428] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 00:37:29.236057 5670 heartbeater.cc:503] Master 127.1.208.62:37203 was elected leader, sending a full tablet report...
W20260808 00:37:29.272214 5449 tablet.cc:2369] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:29.294265 5721 raft_consensus.cc:482] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:29.294425 5721 raft_consensus.cc:504] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.294811 5721 leader_election.cc:288] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427)
I20260808 00:37:29.297741 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.295161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:44725 (local address 127.1.208.1:37067)
0808 00:37:29.295207 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.295208 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.295215 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:29.295243 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.295244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.295254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.295309 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.295483 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.295964 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297052 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.297423 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297485 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.297497 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.297633 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.297671 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:29.297761 5492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.295019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.1.208.3:44725)
0808 00:37:29.295067 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.295080 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.295129 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.295336 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.295337 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.295339 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.295417 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.295420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.296041 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.296042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.296964 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.296967 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.297497 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297497 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.297574 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.297578 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.297621 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.297662 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:37:29.297791 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.295114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:49365 (local address 127.1.208.2:35427)
0808 00:37:29.295162 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.295164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.295349 (+ 185us) server_negotiation.cc:407] Connection header received
0808 00:37:29.295390 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.295392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.295401 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.295448 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.295579 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.296052 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297076 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.297481 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297542 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.297552 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.297688 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.297723 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:29.298007 5722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.295026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35427 (local address 127.1.208.3:49365)
0808 00:37:29.295277 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.295285 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.295342 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.295448 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.295449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.295451 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.295522 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.295523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.296121 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.296122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.296990 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.296993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.297552 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.297553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.297628 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.297631 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.297675 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.297713 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":79,"thread_start_us":114,"threads_started":1}
I20260808 00:37:29.298116 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282b781cfc70453ca7fc0c4b352588af" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" is_pre_election: true
I20260808 00:37:29.298275 5115 raft_consensus.cc:2454] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 0.
I20260808 00:37:29.298339 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282b781cfc70453ca7fc0c4b352588af" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e" is_pre_election: true
I20260808 00:37:29.298501 5362 raft_consensus.cc:2454] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 0.
I20260808 00:37:29.298506 5482 leader_election.cc:302] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.298615 5721 raft_consensus.cc:2781] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:29.298677 5721 raft_consensus.cc:482] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:29.298693 5721 raft_consensus.cc:3037] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.299470 5721 raft_consensus.cc:504] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.299583 5721 leader_election.cc:288] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 election: Requested vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427)
I20260808 00:37:29.299729 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282b781cfc70453ca7fc0c4b352588af" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
I20260808 00:37:29.299806 5115 raft_consensus.cc:3037] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.299793 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "282b781cfc70453ca7fc0c4b352588af" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e"
I20260808 00:37:29.299889 5362 raft_consensus.cc:3037] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.300532 5115 raft_consensus.cc:2454] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 1.
I20260808 00:37:29.300644 5362 raft_consensus.cc:2454] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 1.
I20260808 00:37:29.300693 5482 leader_election.cc:302] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.300839 5721 raft_consensus.cc:2781] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:29.300983 5721 raft_consensus.cc:686] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 1 LEADER]: Becoming Leader. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Running, Role: LEADER
I20260808 00:37:29.301074 5721 consensus_queue.cc:237] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } }
I20260808 00:37:29.301884 4865 catalog_manager.cc:5495] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42e1b25066764f9a9c199bcdda877066 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "42e1b25066764f9a9c199bcdda877066" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } health_report { overall_health: UNKNOWN } } }
Aug 08 00:37:29 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149448, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:29.336030 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.329944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37203 (local address 127.0.0.1:40200)
0808 00:37:29.330082 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.330091 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.330151 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.330276 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.330277 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.330608 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:29.330656 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.330658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.331242 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.331242 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.331761 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.331762 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.332040 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.332041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.332076 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.332563 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:29.332567 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:29.333911 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:29.335184 (+ 1273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.335190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.335192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.335383 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.335523 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:29.335523 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:29.335524 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:29.335574 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:29.335739 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:29.335744 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:29.335781 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.335833 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.335918 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260808 00:37:29.336045 4956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.329992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40200 (local address 127.1.208.62:37203)
0808 00:37:29.330066 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.330068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.330145 (+ 77us) server_negotiation.cc:407] Connection header received
0808 00:37:29.330191 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.330192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.330200 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.330261 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:29.330713 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.331190 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.331816 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.331992 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.332037 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.333986 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:29.333987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:29.333988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:29.333995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:29.335121 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.335438 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.335438 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.335439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.335481 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:29.335611 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:29.335611 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:29.335611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:29.335700 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:29.335742 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.335853 (+ 111us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.335973 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":40}
I20260808 00:37:29.338014 4865 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:40200:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:37:29.338212 4865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:29.339673 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad4dbe8791e4a47833567008fdc2ef9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=110a134681fc467fb0b19ec9ddac98fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:29.339684 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad4dbe8791e4a47833567008fdc2ef9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=110a134681fc467fb0b19ec9ddac98fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:29.339831 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad4dbe8791e4a47833567008fdc2ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.339831 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad4dbe8791e4a47833567008fdc2ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.341002 5707 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: Bootstrap starting.
I20260808 00:37:29.341120 5325 tablet_service.cc:1467] Processing CreateTablet for tablet 6ad4dbe8791e4a47833567008fdc2ef9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=110a134681fc467fb0b19ec9ddac98fd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:29.341310 5325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ad4dbe8791e4a47833567008fdc2ef9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.341480 5708 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: Bootstrap starting.
I20260808 00:37:29.341730 5707 tablet_bootstrap.cc:654] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.341898 5708 tablet_bootstrap.cc:654] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.342334 5707 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: No bootstrap required, opened a new log
I20260808 00:37:29.342384 5707 ts_tablet_manager.cc:1397] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:29.342553 5707 raft_consensus.cc:348] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.342638 5707 raft_consensus.cc:374] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.342655 5707 raft_consensus.cc:729] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.342694 5707 consensus_queue.cc:260] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.342772 5707 ts_tablet_manager.cc:1428] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:29.342779 5708 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: No bootstrap required, opened a new log
I20260808 00:37:29.342826 5708 ts_tablet_manager.cc:1397] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:29.342991 5708 raft_consensus.cc:348] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.343071 5708 raft_consensus.cc:374] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.343091 5708 raft_consensus.cc:729] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.343092 5719 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.343133 5708 consensus_queue.cc:260] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.343182 5719 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.343232 5708 ts_tablet_manager.cc:1428] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:29.343286 5706 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: Bootstrap starting.
I20260808 00:37:29.343937 5706 tablet_bootstrap.cc:654] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.344343 5721 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.344429 5721 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.344537 5706 tablet_bootstrap.cc:492] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: No bootstrap required, opened a new log
I20260808 00:37:29.344583 5706 ts_tablet_manager.cc:1397] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:29.344725 5706 raft_consensus.cc:348] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.344791 5706 raft_consensus.cc:374] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.344810 5706 raft_consensus.cc:729] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.344854 5706 consensus_queue.cc:260] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.344952 5706 ts_tablet_manager.cc:1428] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:29.345007 5718 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.345103 5718 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.356838 5719 raft_consensus.cc:482] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:29.356938 5719 raft_consensus.cc:504] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.357334 5719 leader_election.cc:288] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:29.360451 5705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.357552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:37931 (local address 127.1.208.2:35427)
0808 00:37:29.357601 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.357603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.357649 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:37:29.357711 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.357713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.357722 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.357775 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.357933 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.358484 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.359527 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.360107 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.360170 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.360181 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.360347 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.360383 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:29.360472 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.357546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:59091 (local address 127.1.208.3:41829)
0808 00:37:29.357594 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.357596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.357739 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:37:29.357789 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.357790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.357801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.357849 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:29.357980 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.358508 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.359549 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.360092 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.360170 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.360181 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.360344 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.360381 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:29.360491 5039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.357463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35427 (local address 127.1.208.1:37931)
0808 00:37:29.357600 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.357611 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.357664 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.357784 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.357785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.357786 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.357862 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.357864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.358552 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.358553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.359431 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.359434 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.360174 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.360175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.360272 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.360279 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.360342 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.360377 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":35}
I20260808 00:37:29.360724 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad4dbe8791e4a47833567008fdc2ef9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e" is_pre_election: true
I20260808 00:37:29.360811 5362 raft_consensus.cc:2454] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 0.
I20260808 00:37:29.360832 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.357498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41829 (local address 127.1.208.1:59091)
0808 00:37:29.357667 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.357677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.357734 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.357852 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.357853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.357854 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:29.357920 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.357922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.358579 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.358580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.359432 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.359435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.360191 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.360192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.360272 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.360280 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.360333 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.360376 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":10,"thread_start_us":110,"threads_started":1}
I20260808 00:37:29.360988 5026 leader_election.cc:302] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42fe980ab4774997b4a67e6c95b3927e, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.361127 5719 raft_consensus.cc:2781] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:29.361173 5719 raft_consensus.cc:482] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:29.361195 5719 raft_consensus.cc:3037] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.361147 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad4dbe8791e4a47833567008fdc2ef9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066" is_pre_election: true
I20260808 00:37:29.361251 5591 raft_consensus.cc:2454] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 0.
I20260808 00:37:29.361974 5719 raft_consensus.cc:504] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.362097 5719 leader_election.cc:288] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 election: Requested vote from peers 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:29.362229 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad4dbe8791e4a47833567008fdc2ef9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e"
I20260808 00:37:29.362247 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ad4dbe8791e4a47833567008fdc2ef9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066"
I20260808 00:37:29.362303 5362 raft_consensus.cc:3037] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.362319 5591 raft_consensus.cc:3037] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.362893 5362 raft_consensus.cc:2454] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 1.
I20260808 00:37:29.362939 5591 raft_consensus.cc:2454] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 1.
I20260808 00:37:29.363065 5026 leader_election.cc:302] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42fe980ab4774997b4a67e6c95b3927e, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.363157 5719 raft_consensus.cc:2781] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:29.363320 5719 raft_consensus.cc:686] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Running, Role: LEADER
I20260808 00:37:29.363413 5719 consensus_queue.cc:237] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.363853 5750 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: TxnStatusTablet state changed. Reason: New leader 6c672a34b6e94d5aa1212e68d2f165f2. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.363973 5750 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:29.364172 5753 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:37:29.364183 4861 catalog_manager.cc:5495] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1). New cstate: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:29.364357 5753 txn_status_manager.cc:929] Loading transaction status metadata into memory...
W20260808 00:37:29.366855 5208 tablet.cc:2369] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:29.375663 5738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.373000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.0.0.1:43702)
0808 00:37:29.373075 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.373087 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.373139 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.373321 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.373322 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.373324 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:29.373374 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.373376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.374251 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.374252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.374811 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.374812 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.375241 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.375242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.375283 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.375297 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.375545 (+ 248us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.375548 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.375571 (+ 23us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.375605 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:37:29.375695 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.373091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43702 (local address 127.1.208.1:37067)
0808 00:37:29.373152 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.373154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.373184 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:37:29.373213 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.373214 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.373232 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.373330 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.373432 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.374180 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.374881 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.375180 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.375247 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.375370 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.375491 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.375567 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.375582 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.375604 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:37:29.377458 5591 raft_consensus.cc:1261] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Refusing update from remote peer 6c672a34b6e94d5aa1212e68d2f165f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.377476 5362 raft_consensus.cc:1261] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Refusing update from remote peer 6c672a34b6e94d5aa1212e68d2f165f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.377764 5750 consensus_queue.cc:1035] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:29.377894 5719 consensus_queue.cc:1035] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:29.379155 5721 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: TxnStatusTablet state changed. Reason: New leader 6c672a34b6e94d5aa1212e68d2f165f2. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.379156 5718 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: TxnStatusTablet state changed. Reason: New leader 6c672a34b6e94d5aa1212e68d2f165f2. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.379246 5721 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.379253 5718 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.380137 5750 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.380223 5750 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:29.380335 5758 mvcc.cc:205] Tried to move back new op lower bound from 7316068144647544832 to 7316068144593227776. Current Snapshot: MvccSnapshot[applied={T|T < 7316068144647544832}]
I20260808 00:37:29.380643 5721 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.380722 5721 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42e1b25066764f9a9c199bcdda877066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.381379 5765 mvcc.cc:205] Tried to move back new op lower bound from 7316068144647544832 to 7316068144593227776. Current Snapshot: MvccSnapshot[applied={T|T < 7316068144647544832 or (T in {7316068144647544832})}]
I20260808 00:37:29.381832 5719 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.381909 5719 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 6c672a34b6e94d5aa1212e68d2f165f2: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:29.382442 5718 tablet_replica.cc:439] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } } }
I20260808 00:37:29.382527 5718 tablet_replica.cc:442] T 6ad4dbe8791e4a47833567008fdc2ef9 P 42fe980ab4774997b4a67e6c95b3927e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:29.382666 5764 mvcc.cc:205] Tried to move back new op lower bound from 7316068144647544832 to 7316068144593227776. Current Snapshot: MvccSnapshot[applied={T|T < 7316068144647544832}]
I20260808 00:37:29.385254 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.382748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53892 (local address 127.1.208.3:41829)
0808 00:37:29.382827 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.382828 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.382836 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:29.382876 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.382877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.382893 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.382963 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.383130 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.383688 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.384488 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.384709 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.384763 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.384925 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.385048 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.385114 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.385157 (+ 43us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.385180 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 00:37:29.390957 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.387640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.1.208.3:56865)
0808 00:37:29.387766 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.387776 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.387818 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.387935 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.387936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.387936 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:29.387987 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.387988 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.389617 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.389618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.390188 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.390190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.390616 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.390616 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.390662 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.390676 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.390848 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.390849 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.390876 (+ 27us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.390908 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 00:37:29.390970 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.387707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:56865 (local address 127.1.208.1:37067)
0808 00:37:29.387750 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.387752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.387810 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:37:29.387864 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.387865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.387875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.387924 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.388034 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.389543 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.390270 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.390547 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.390619 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.390764 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.390798 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.390839 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.390881 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.390899 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:29.393376 5362 raft_consensus.cc:1261] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Refusing update from remote peer 42e1b25066764f9a9c199bcdda877066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.393410 5115 raft_consensus.cc:1261] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Refusing update from remote peer 42e1b25066764f9a9c199bcdda877066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.393649 5721 consensus_queue.cc:1035] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:29.393791 5723 consensus_queue.cc:1035] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:29.402431 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.399721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:37613 (local address 127.1.208.3:41829)
0808 00:37:29.399765 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.399767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.399926 (+ 159us) server_negotiation.cc:407] Connection header received
0808 00:37:29.399984 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.399985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.399995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.400043 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.400203 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.400633 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.401789 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.402028 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.402078 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.402261 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.402285 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.402316 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.402362 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.402377 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:29.402494 5789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.399686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41829 (local address 127.1.208.1:37613)
0808 00:37:29.399865 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.399876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.399940 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.400052 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.400054 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.400070 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:29.400137 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.400138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.400694 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.400695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.401678 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.401682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.402096 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.402097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.402170 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.402188 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.402319 (+ 131us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.402321 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.402356 (+ 35us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.402397 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260808 00:37:29.411862 4865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:29.411983 4865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:29.412770 4865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Sending DeleteTablet for 3 replicas of tablet 282b781cfc70453ca7fc0c4b352588af
I20260808 00:37:29.413076 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet 282b781cfc70453ca7fc0c4b352588af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:29 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:43686
I20260808 00:37:29.413199 5325 tablet_service.cc:1514] Processing DeleteTablet for tablet 282b781cfc70453ca7fc0c4b352588af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:29 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33988
I20260808 00:37:29.413529 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 282b781cfc70453ca7fc0c4b352588af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:29 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:53880
I20260808 00:37:29.413782 4865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:29.413937 4865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:29.414242 5794 tablet_replica.cc:330] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: stopping tablet replica
I20260808 00:37:29.414362 5794 raft_consensus.cc:2229] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:29.415086 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 2f11a68e6eac447091c51f877149aeb9 (DEFAULT_TABLE table=test-table [id=be9d6040b88848c7b67032b49f310af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.415220 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f11a68e6eac447091c51f877149aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.415222 5325 tablet_service.cc:1467] Processing CreateTablet for tablet 2f11a68e6eac447091c51f877149aeb9 (DEFAULT_TABLE table=test-table [id=be9d6040b88848c7b67032b49f310af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.415359 5325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f11a68e6eac447091c51f877149aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.416700 5707 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Bootstrap starting.
I20260808 00:37:29.416735 5706 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Bootstrap starting.
I20260808 00:37:29.417150 5706 tablet_bootstrap.cc:654] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.417150 5707 tablet_bootstrap.cc:654] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.417557 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 2f11a68e6eac447091c51f877149aeb9 (DEFAULT_TABLE table=test-table [id=be9d6040b88848c7b67032b49f310af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:29.417672 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f11a68e6eac447091c51f877149aeb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:29.419451 5708 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Bootstrap starting.
I20260808 00:37:29.420239 5794 raft_consensus.cc:2258] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:29.420499 5708 tablet_bootstrap.cc:654] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:29.420783 5794 ts_tablet_manager.cc:1905] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:29.423522 5794 ts_tablet_manager.cc:1918] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:29.423534 5792 tablet_replica.cc:330] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: stopping tablet replica
I20260808 00:37:29.423709 5794 log.cc:1196] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/282b781cfc70453ca7fc0c4b352588af
I20260808 00:37:29.423659 5792 raft_consensus.cc:2229] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:29.424125 5794 ts_tablet_manager.cc:1939] T 282b781cfc70453ca7fc0c4b352588af P 42fe980ab4774997b4a67e6c95b3927e: Deleting consensus metadata
I20260808 00:37:29.424142 5792 raft_consensus.cc:2258] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:29.424638 4829 catalog_manager.cc:4841] TS 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427): tablet 282b781cfc70453ca7fc0c4b352588af (table test-table [id=6a90eb5138a2492e89dc7d0348092d26]) successfully deleted
I20260808 00:37:29.424800 5792 ts_tablet_manager.cc:1905] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:29.424903 5795 tablet_replica.cc:330] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: stopping tablet replica
I20260808 00:37:29.425015 5795 raft_consensus.cc:2229] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:29.425124 5795 raft_consensus.cc:2258] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:29.425875 5795 ts_tablet_manager.cc:1905] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:29.427235 5706 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: No bootstrap required, opened a new log
I20260808 00:37:29.427291 5706 ts_tablet_manager.cc:1397] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.001s
I20260808 00:37:29.427656 5708 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: No bootstrap required, opened a new log
I20260808 00:37:29.427706 5708 ts_tablet_manager.cc:1397] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260808 00:37:29.427646 5706 raft_consensus.cc:348] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.427733 5706 raft_consensus.cc:374] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.427754 5706 raft_consensus.cc:729] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.427798 5706 consensus_queue.cc:260] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.427878 5706 ts_tablet_manager.cc:1428] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:29.428125 5708 raft_consensus.cc:348] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.428200 5708 raft_consensus.cc:374] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.428220 5708 raft_consensus.cc:729] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.428262 5708 consensus_queue.cc:260] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.428349 5708 ts_tablet_manager.cc:1428] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:29.428618 5795 ts_tablet_manager.cc:1918] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:29.428681 5795 log.cc:1196] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/282b781cfc70453ca7fc0c4b352588af
I20260808 00:37:29.428913 5795 ts_tablet_manager.cc:1939] T 282b781cfc70453ca7fc0c4b352588af P 42e1b25066764f9a9c199bcdda877066: Deleting consensus metadata
I20260808 00:37:29.429189 5707 tablet_bootstrap.cc:492] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: No bootstrap required, opened a new log
I20260808 00:37:29.429246 5707 ts_tablet_manager.cc:1397] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.001s
I20260808 00:37:29.429481 5707 raft_consensus.cc:348] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.429562 5707 raft_consensus.cc:374] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:29.429584 5707 raft_consensus.cc:729] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Initialized, Role: FOLLOWER
I20260808 00:37:29.429626 5707 consensus_queue.cc:260] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.429788 5707 ts_tablet_manager.cc:1428] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:29.430365 4832 catalog_manager.cc:4841] TS 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829): tablet 282b781cfc70453ca7fc0c4b352588af (table test-table [id=6a90eb5138a2492e89dc7d0348092d26]) successfully deleted
I20260808 00:37:29.429795 5792 ts_tablet_manager.cc:1918] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:29.431020 5792 log.cc:1196] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/282b781cfc70453ca7fc0c4b352588af
I20260808 00:37:29.431298 5792 ts_tablet_manager.cc:1939] T 282b781cfc70453ca7fc0c4b352588af P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting consensus metadata
I20260808 00:37:29.431707 4831 catalog_manager.cc:4841] TS 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067): tablet 282b781cfc70453ca7fc0c4b352588af (table test-table [id=6a90eb5138a2492e89dc7d0348092d26]) successfully deleted
W20260808 00:37:29.431747 5671 tablet.cc:2369] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:29.583562 5750 raft_consensus.cc:482] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:29.583684 5750 raft_consensus.cc:504] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.583858 5750 leader_election.cc:288] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:29.584093 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f11a68e6eac447091c51f877149aeb9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e" is_pre_election: true
I20260808 00:37:29.584091 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f11a68e6eac447091c51f877149aeb9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066" is_pre_election: true
I20260808 00:37:29.584203 5362 raft_consensus.cc:2454] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 0.
I20260808 00:37:29.584219 5591 raft_consensus.cc:2454] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 0.
I20260808 00:37:29.584394 5029 leader_election.cc:302] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 42fe980ab4774997b4a67e6c95b3927e, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.584520 5750 raft_consensus.cc:2781] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:29.584586 5750 raft_consensus.cc:482] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:29.584607 5750 raft_consensus.cc:3037] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.585367 5750 raft_consensus.cc:504] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.585500 5750 leader_election.cc:288] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 election: Requested vote from peers 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:29.585654 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f11a68e6eac447091c51f877149aeb9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066"
I20260808 00:37:29.585657 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f11a68e6eac447091c51f877149aeb9" candidate_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e"
I20260808 00:37:29.585721 5591 raft_consensus.cc:3037] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.585729 5362 raft_consensus.cc:3037] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:29.586349 5591 raft_consensus.cc:2454] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 1.
I20260808 00:37:29.586357 5362 raft_consensus.cc:2454] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c672a34b6e94d5aa1212e68d2f165f2 in term 1.
I20260808 00:37:29.586535 5029 leader_election.cc:302] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:29.586651 5750 raft_consensus.cc:2781] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:29.586714 5750 raft_consensus.cc:686] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 LEADER]: Becoming Leader. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Running, Role: LEADER
I20260808 00:37:29.586766 5750 consensus_queue.cc:237] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:29.587325 4861 catalog_manager.cc:5495] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1). New cstate: current_term: 1 leader_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:29.609642 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.607687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43704 (local address 127.1.208.1:37067)
0808 00:37:29.607729 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.607731 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.607740 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:29.607799 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.607799 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.607807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.607849 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.607945 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.608427 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.609134 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.609327 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.609374 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.609480 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.609506 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.609536 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.609569 (+ 33us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.609582 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:29.613523 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.611787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:57851 (local address 127.1.208.1:37067)
0808 00:37:29.611837 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:29.611838 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:29.611846 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:29.611873 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:29.611873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:29.611880 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:29.611912 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:29.611984 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.612402 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.613021 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.613181 (+ 160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.613219 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.613327 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.613354 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.613388 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:29.613432 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 00:37:29.613450 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":24}
I20260808 00:37:29.613572 5789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:29.611732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.1.208.1:57851)
0808 00:37:29.611782 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:29.611793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:29.611832 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:29.611916 (+ 84us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:29.611917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:29.611917 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:29.611950 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.611952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.612448 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.612448 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.612970 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:29.612972 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:29.613224 (+ 252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:29.613225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:29.613257 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:29.613262 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:29.613388 (+ 126us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:29.613389 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:29.613430 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 00:37:29.613472 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":1}
I20260808 00:37:29.615336 5591 raft_consensus.cc:1261] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Refusing update from remote peer 6c672a34b6e94d5aa1212e68d2f165f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.615339 5362 raft_consensus.cc:1261] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Refusing update from remote peer 6c672a34b6e94d5aa1212e68d2f165f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:29.615538 5719 consensus_queue.cc:1035] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:29.615649 5750 consensus_queue.cc:1035] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:37:29.619988 5758 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:37:31.138236 4865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:31.138353 4865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:31.139241 4865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Sending DeleteTablet for 3 replicas of tablet 2f11a68e6eac447091c51f877149aeb9
I20260808 00:37:31.139537 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f11a68e6eac447091c51f877149aeb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:43686
I20260808 00:37:31.139549 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f11a68e6eac447091c51f877149aeb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:53880
I20260808 00:37:31.139566 5325 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f11a68e6eac447091c51f877149aeb9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33988
I20260808 00:37:31.143975 5843 tablet_replica.cc:330] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: stopping tablet replica
I20260808 00:37:31.144048 5843 raft_consensus.cc:2229] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:31.144158 5844 tablet_replica.cc:330] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: stopping tablet replica
I20260808 00:37:31.144220 5844 raft_consensus.cc:2229] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:31.144261 5844 raft_consensus.cc:2258] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.144344 5843 raft_consensus.cc:2258] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.144369 5841 tablet_replica.cc:330] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: stopping tablet replica
I20260808 00:37:31.144443 5841 raft_consensus.cc:2229] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:31.144501 5841 raft_consensus.cc:2258] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.144707 5843 ts_tablet_manager.cc:1905] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.144726 5841 ts_tablet_manager.cc:1905] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.144745 5844 ts_tablet_manager.cc:1905] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.146587 5843 ts_tablet_manager.cc:1918] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:37:31.146587 5841 ts_tablet_manager.cc:1918] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:37:31.146649 5841 log.cc:1196] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/2f11a68e6eac447091c51f877149aeb9
I20260808 00:37:31.146646 5844 ts_tablet_manager.cc:1918] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:37:31.146699 5844 log.cc:1196] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/2f11a68e6eac447091c51f877149aeb9
I20260808 00:37:31.146770 5843 log.cc:1196] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/2f11a68e6eac447091c51f877149aeb9
I20260808 00:37:31.146891 5841 ts_tablet_manager.cc:1939] T 2f11a68e6eac447091c51f877149aeb9 P 42fe980ab4774997b4a67e6c95b3927e: Deleting consensus metadata
I20260808 00:37:31.146921 5844 ts_tablet_manager.cc:1939] T 2f11a68e6eac447091c51f877149aeb9 P 42e1b25066764f9a9c199bcdda877066: Deleting consensus metadata
I20260808 00:37:31.147092 5843 ts_tablet_manager.cc:1939] T 2f11a68e6eac447091c51f877149aeb9 P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting consensus metadata
I20260808 00:37:31.147313 4832 catalog_manager.cc:4841] TS 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829): tablet 2f11a68e6eac447091c51f877149aeb9 (table test-table [id=be9d6040b88848c7b67032b49f310af1]) successfully deleted
I20260808 00:37:31.147374 4829 catalog_manager.cc:4841] TS 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427): tablet 2f11a68e6eac447091c51f877149aeb9 (table test-table [id=be9d6040b88848c7b67032b49f310af1]) successfully deleted
I20260808 00:37:31.147428 4831 catalog_manager.cc:4841] TS 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067): tablet 2f11a68e6eac447091c51f877149aeb9 (table test-table [id=be9d6040b88848c7b67032b49f310af1]) successfully deleted
Aug 08 00:37:31 dist-test-slave-gjw7 krb5kdc[4781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149451, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:37:31 dist-test-slave-gjw7 krb5kdc[4781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149451, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:31.169600 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.158633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43920 (local address 127.1.208.62:37203)
0808 00:37:31.158792 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.158794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.158801 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:31.158830 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.158830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.158841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.158896 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:31.159449 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.159997 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.160748 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.160948 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.161007 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.167203 (+ 6196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:31.167205 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:31.167205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:31.167214 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:31.168475 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:31.168779 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:31.168780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:31.168781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:31.168838 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:31.169061 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:31.169062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:31.169063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:31.169179 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:31.169271 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.169396 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.169489 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260808 00:37:31.171061 4865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:31.171236 4865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:31.172578 5325 tablet_service.cc:1467] Processing CreateTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (DEFAULT_TABLE table=test-table [id=8961480e3a5342778b944936cc99e812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.172583 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (DEFAULT_TABLE table=test-table [id=8961480e3a5342778b944936cc99e812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.172583 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (DEFAULT_TABLE table=test-table [id=8961480e3a5342778b944936cc99e812]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.172722 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.172724 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.172724 5325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.174182 5859 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Bootstrap starting.
I20260808 00:37:31.174438 5858 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Bootstrap starting.
I20260808 00:37:31.174543 5859 tablet_bootstrap.cc:654] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.174666 5860 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Bootstrap starting.
I20260808 00:37:31.175100 5858 tablet_bootstrap.cc:654] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.175330 5860 tablet_bootstrap.cc:654] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.175345 5859 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: No bootstrap required, opened a new log
I20260808 00:37:31.175388 5859 ts_tablet_manager.cc:1397] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:31.175506 5859 raft_consensus.cc:348] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.175590 5859 raft_consensus.cc:374] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.175611 5859 raft_consensus.cc:729] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.175694 5859 consensus_queue.cc:260] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.175899 5858 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: No bootstrap required, opened a new log
I20260808 00:37:31.175894 5859 ts_tablet_manager.cc:1428] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:31.175961 5858 ts_tablet_manager.cc:1397] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:31.176191 5858 raft_consensus.cc:348] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.176277 5858 raft_consensus.cc:374] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.176299 5858 raft_consensus.cc:729] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.176353 5858 consensus_queue.cc:260] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.176576 5858 ts_tablet_manager.cc:1428] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:31.176800 5860 tablet_bootstrap.cc:492] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: No bootstrap required, opened a new log
I20260808 00:37:31.176851 5860 ts_tablet_manager.cc:1397] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:31.176962 5860 raft_consensus.cc:348] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.177034 5860 raft_consensus.cc:374] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.177068 5860 raft_consensus.cc:729] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.177110 5860 consensus_queue.cc:260] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.177208 5860 ts_tablet_manager.cc:1428] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:31.184892 5865 raft_consensus.cc:482] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:31.184966 5865 raft_consensus.cc:504] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.185112 5865 leader_election.cc:288] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427)
I20260808 00:37:31.185259 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b17c01b73d1470a9b99ebf5cca8bb1b" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" is_pre_election: true
I20260808 00:37:31.185273 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b17c01b73d1470a9b99ebf5cca8bb1b" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e" is_pre_election: true
I20260808 00:37:31.185387 5115 raft_consensus.cc:2454] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 0.
I20260808 00:37:31.185384 5362 raft_consensus.cc:2454] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 0.
I20260808 00:37:31.185549 5482 leader_election.cc:302] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:31.185669 5865 raft_consensus.cc:2781] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:31.185712 5865 raft_consensus.cc:482] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:31.185725 5865 raft_consensus.cc:3037] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.186494 5865 raft_consensus.cc:504] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.186614 5865 leader_election.cc:288] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 election: Requested vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427)
I20260808 00:37:31.186753 5362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b17c01b73d1470a9b99ebf5cca8bb1b" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42fe980ab4774997b4a67e6c95b3927e"
I20260808 00:37:31.186836 5362 raft_consensus.cc:3037] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.187387 5362 raft_consensus.cc:2454] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 1.
I20260808 00:37:31.187502 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b17c01b73d1470a9b99ebf5cca8bb1b" candidate_uuid: "42e1b25066764f9a9c199bcdda877066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
I20260808 00:37:31.187546 5481 leader_election.cc:302] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 42fe980ab4774997b4a67e6c95b3927e; no voters:
I20260808 00:37:31.187588 5115 raft_consensus.cc:3037] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.187635 5865 raft_consensus.cc:2781] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:31.187790 5865 raft_consensus.cc:686] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 1 LEADER]: Becoming Leader. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Running, Role: LEADER
I20260808 00:37:31.187865 5865 consensus_queue.cc:237] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.188334 5115 raft_consensus.cc:2454] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42e1b25066764f9a9c199bcdda877066 in term 1.
I20260808 00:37:31.188446 4861 catalog_manager.cc:5495] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 reported cstate change: term changed from 0 to 1, leader changed from <none> to 42e1b25066764f9a9c199bcdda877066 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "42e1b25066764f9a9c199bcdda877066" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } health_report { overall_health: HEALTHY } } }
I20260808 00:37:31.199240 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.197166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44388 (local address 127.1.208.3:41829)
0808 00:37:31.197318 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.197320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.197327 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:31.197351 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.197352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.197364 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.197429 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:31.197526 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.198001 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.198712 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.198900 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.198956 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.199066 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.199091 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.199120 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.199168 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.199183 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260808 00:37:31.201159 5362 raft_consensus.cc:1261] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Refusing update from remote peer 42e1b25066764f9a9c199bcdda877066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:31.201215 5115 raft_consensus.cc:1261] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Refusing update from remote peer 42e1b25066764f9a9c199bcdda877066: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:31.201414 5868 consensus_queue.cc:1035] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:31.201520 5865 consensus_queue.cc:1035] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:31.211350 4865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:31.211458 4865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:31.211999 4865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Sending DeleteTablet for 3 replicas of tablet 3b17c01b73d1470a9b99ebf5cca8bb1b
I20260808 00:37:31.212229 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:53880
I20260808 00:37:31.212225 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:43686
I20260808 00:37:31.212327 5844 tablet_replica.cc:330] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: stopping tablet replica
I20260808 00:37:31.212342 5325 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b17c01b73d1470a9b99ebf5cca8bb1b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:31 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33988
I20260808 00:37:31.212378 5844 raft_consensus.cc:2229] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:31.212435 5844 raft_consensus.cc:2258] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.212435 5843 tablet_replica.cc:330] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: stopping tablet replica
I20260808 00:37:31.212445 5841 tablet_replica.cc:330] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: stopping tablet replica
I20260808 00:37:31.212486 5843 raft_consensus.cc:2229] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:31.212498 5841 raft_consensus.cc:2229] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:31.212523 5843 raft_consensus.cc:2258] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.212532 5841 raft_consensus.cc:2258] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:31.212627 4865 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:31.212767 4865 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:31.212821 5844 ts_tablet_manager.cc:1905] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.212852 5843 ts_tablet_manager.cc:1905] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.212860 5841 ts_tablet_manager.cc:1905] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:31.214599 5843 ts_tablet_manager.cc:1918] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:31.214617 5841 ts_tablet_manager.cc:1918] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:31.214661 5843 log.cc:1196] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/3b17c01b73d1470a9b99ebf5cca8bb1b
I20260808 00:37:31.214668 5841 log.cc:1196] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/3b17c01b73d1470a9b99ebf5cca8bb1b
I20260808 00:37:31.214881 5841 ts_tablet_manager.cc:1939] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42fe980ab4774997b4a67e6c95b3927e: Deleting consensus metadata
I20260808 00:37:31.214897 5843 ts_tablet_manager.cc:1939] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 6c672a34b6e94d5aa1212e68d2f165f2: Deleting consensus metadata
I20260808 00:37:31.215139 5844 ts_tablet_manager.cc:1918] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:31.215163 4829 catalog_manager.cc:4841] TS 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427): tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (table test-table [id=8961480e3a5342778b944936cc99e812]) successfully deleted
I20260808 00:37:31.215197 5844 log.cc:1196] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/3b17c01b73d1470a9b99ebf5cca8bb1b
I20260808 00:37:31.215236 5325 tablet_service.cc:1467] Processing CreateTablet for tablet f88b2eae176c4057a784ff6d350f6190 (DEFAULT_TABLE table=test-table [id=3633fc99dc18423686ad8916b4691bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.215317 4831 catalog_manager.cc:4841] TS 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067): tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (table test-table [id=8961480e3a5342778b944936cc99e812]) successfully deleted
I20260808 00:37:31.215333 5325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88b2eae176c4057a784ff6d350f6190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.215400 5844 ts_tablet_manager.cc:1939] T 3b17c01b73d1470a9b99ebf5cca8bb1b P 42e1b25066764f9a9c199bcdda877066: Deleting consensus metadata
I20260808 00:37:31.215423 5105 tablet_service.cc:1467] Processing CreateTablet for tablet f88b2eae176c4057a784ff6d350f6190 (DEFAULT_TABLE table=test-table [id=3633fc99dc18423686ad8916b4691bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.215505 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88b2eae176c4057a784ff6d350f6190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.215683 4832 catalog_manager.cc:4841] TS 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829): tablet 3b17c01b73d1470a9b99ebf5cca8bb1b (table test-table [id=8961480e3a5342778b944936cc99e812]) successfully deleted
I20260808 00:37:31.215772 5562 tablet_service.cc:1467] Processing CreateTablet for tablet f88b2eae176c4057a784ff6d350f6190 (DEFAULT_TABLE table=test-table [id=3633fc99dc18423686ad8916b4691bbe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:31.215874 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f88b2eae176c4057a784ff6d350f6190. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:31.216462 5858 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Bootstrap starting.
I20260808 00:37:31.216691 5860 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2: Bootstrap starting.
I20260808 00:37:31.216786 5858 tablet_bootstrap.cc:654] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.216984 5860 tablet_bootstrap.cc:654] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.217197 5859 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Bootstrap starting.
I20260808 00:37:31.217636 5859 tablet_bootstrap.cc:654] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:31.217872 5860 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2: No bootstrap required, opened a new log
I20260808 00:37:31.217916 5860 ts_tablet_manager.cc:1397] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:31.218036 5860 raft_consensus.cc:348] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218096 5860 raft_consensus.cc:374] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.218107 5860 raft_consensus.cc:729] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c672a34b6e94d5aa1212e68d2f165f2, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.218143 5860 consensus_queue.cc:260] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218235 5860 ts_tablet_manager.cc:1428] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:31.218343 5859 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: No bootstrap required, opened a new log
I20260808 00:37:31.218394 5859 ts_tablet_manager.cc:1397] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:31.218446 5858 tablet_bootstrap.cc:492] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: No bootstrap required, opened a new log
I20260808 00:37:31.218489 5858 ts_tablet_manager.cc:1397] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:31.218494 5859 raft_consensus.cc:348] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218564 5859 raft_consensus.cc:374] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.218590 5859 raft_consensus.cc:729] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42e1b25066764f9a9c199bcdda877066, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.218632 5859 consensus_queue.cc:260] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218633 5858 raft_consensus.cc:348] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218691 5858 raft_consensus.cc:374] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:31.218720 5858 raft_consensus.cc:729] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Initialized, Role: FOLLOWER
I20260808 00:37:31.218719 5859 ts_tablet_manager.cc:1428] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:31.218757 5858 consensus_queue.cc:260] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.218858 5858 ts_tablet_manager.cc:1428] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:31.395113 5867 raft_consensus.cc:482] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:31.395221 5867 raft_consensus.cc:504] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.395601 5867 leader_election.cc:288] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:31.399362 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.395871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:55833 (local address 127.1.208.1:37067)
0808 00:37:31.396034 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.396036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.396168 (+ 132us) server_negotiation.cc:407] Connection header received
0808 00:37:31.396230 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.396231 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.396241 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.396315 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:31.396670 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.397155 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.398544 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.398965 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.399030 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.399042 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.399241 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.399284 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260808 00:37:31.399442 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.395888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:40491 (local address 127.1.208.3:41829)
0808 00:37:31.395976 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.395978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.396048 (+ 70us) server_negotiation.cc:407] Connection header received
0808 00:37:31.396169 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.396170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.396182 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.396227 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:31.396651 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.397158 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.398555 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.398963 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.399029 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.399044 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.399313 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.399366 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 00:37:31.399497 5886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.395783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.1.208.2:55833)
0808 00:37:31.396096 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:31.396106 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:31.396177 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:31.396328 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:31.396329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:31.396332 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:31.396587 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.396590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.397225 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.397226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.398456 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.398458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.399040 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.399041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.399170 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.399174 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:31.399224 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:37:31.399270 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":80,"thread_start_us":216,"threads_started":1}
I20260808 00:37:31.399511 5884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.395780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41829 (local address 127.1.208.2:40491)
0808 00:37:31.395973 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:31.395984 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:31.396078 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:31.396259 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:31.396260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:31.396262 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:31.396527 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.396531 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.397491 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.397493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.398448 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.398451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.399037 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.399039 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.399197 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.399201 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:31.399289 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:37:31.399334 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":109,"threads_started":1}
I20260808 00:37:31.399816 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88b2eae176c4057a784ff6d350f6190" candidate_uuid: "42fe980ab4774997b4a67e6c95b3927e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" is_pre_election: true
I20260808 00:37:31.399853 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88b2eae176c4057a784ff6d350f6190" candidate_uuid: "42fe980ab4774997b4a67e6c95b3927e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066" is_pre_election: true
I20260808 00:37:31.399916 5115 raft_consensus.cc:2454] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42fe980ab4774997b4a67e6c95b3927e in term 0.
I20260808 00:37:31.399945 5591 raft_consensus.cc:2454] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 42fe980ab4774997b4a67e6c95b3927e in term 0.
I20260808 00:37:31.400097 5245 leader_election.cc:302] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42fe980ab4774997b4a67e6c95b3927e, 6c672a34b6e94d5aa1212e68d2f165f2; no voters:
I20260808 00:37:31.400259 5867 raft_consensus.cc:2781] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:31.400295 5867 raft_consensus.cc:482] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:31.400315 5867 raft_consensus.cc:3037] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.401000 5867 raft_consensus.cc:504] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.401129 5867 leader_election.cc:288] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [CANDIDATE]: Term 1 election: Requested vote from peers 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067), 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829)
I20260808 00:37:31.401271 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88b2eae176c4057a784ff6d350f6190" candidate_uuid: "42fe980ab4774997b4a67e6c95b3927e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42e1b25066764f9a9c199bcdda877066"
I20260808 00:37:31.401372 5591 raft_consensus.cc:3037] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.401348 5115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f88b2eae176c4057a784ff6d350f6190" candidate_uuid: "42fe980ab4774997b4a67e6c95b3927e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c672a34b6e94d5aa1212e68d2f165f2"
I20260808 00:37:31.401445 5115 raft_consensus.cc:3037] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:31.401993 5591 raft_consensus.cc:2454] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42fe980ab4774997b4a67e6c95b3927e in term 1.
I20260808 00:37:31.402176 5246 leader_election.cc:302] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42e1b25066764f9a9c199bcdda877066, 42fe980ab4774997b4a67e6c95b3927e; no voters:
I20260808 00:37:31.402288 5867 raft_consensus.cc:2781] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:31.402334 5115 raft_consensus.cc:2454] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42fe980ab4774997b4a67e6c95b3927e in term 1.
I20260808 00:37:31.402550 5867 raft_consensus.cc:686] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 1 LEADER]: Becoming Leader. State: Replica: 42fe980ab4774997b4a67e6c95b3927e, State: Running, Role: LEADER
I20260808 00:37:31.402631 5867 consensus_queue.cc:237] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } }
I20260808 00:37:31.403460 4861 catalog_manager.cc:5495] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e reported cstate change: term changed from 0 to 1, leader changed from <none> to 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2). New cstate: current_term: 1 leader_uuid: "42fe980ab4774997b4a67e6c95b3927e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42fe980ab4774997b4a67e6c95b3927e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:31.447438 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.444753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.1.208.2:35427)
0808 00:37:31.445078 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.445080 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.445086 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:37:31.445150 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.445150 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.445171 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.445239 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:31.445414 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.446010 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.446684 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.446945 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.446996 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.447090 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.447202 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.447250 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.447266 (+ 16us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.447282 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":205,"threads_started":1}
I20260808 00:37:31.452173 5885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.450221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:42137 (local address 127.1.208.1:37067)
0808 00:37:31.450277 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.450279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.450340 (+ 61us) server_negotiation.cc:407] Connection header received
0808 00:37:31.450385 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.450386 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.450396 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.450453 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:31.450550 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.450990 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.451654 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.451836 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.451880 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.452009 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.452032 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.452063 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.452100 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.452116 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:31.452191 5892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.450066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.1.208.2:42137)
0808 00:37:31.450276 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:31.450290 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:31.450348 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:31.450459 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:31.450460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:31.450461 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:31.450508 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.450509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.451040 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.451040 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.451599 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.451601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.451899 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.451900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.451940 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.451953 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.452068 (+ 115us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.452069 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:31.452097 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 00:37:31.452128 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":130,"threads_started":1}
I20260808 00:37:31.454051 5115 raft_consensus.cc:1261] T f88b2eae176c4057a784ff6d350f6190 P 6c672a34b6e94d5aa1212e68d2f165f2 [term 1 FOLLOWER]: Refusing update from remote peer 42fe980ab4774997b4a67e6c95b3927e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:31.454079 5591 raft_consensus.cc:1261] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Refusing update from remote peer 42fe980ab4774997b4a67e6c95b3927e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:31.454339 5887 consensus_queue.cc:1035] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [LEADER]: Connected to new peer: Peer: permanent_uuid: "42e1b25066764f9a9c199bcdda877066" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:31.454468 5867 consensus_queue.cc:1035] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c672a34b6e94d5aa1212e68d2f165f2" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:31.461454 5889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.459022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:51257 (local address 127.1.208.2:35427)
0808 00:37:31.459056 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:31.459058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:31.459255 (+ 197us) server_negotiation.cc:407] Connection header received
0808 00:37:31.459278 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:31.459278 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:31.459287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:31.459337 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:31.459439 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.459915 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.460775 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.461020 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.461071 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.461208 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.461245 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.461306 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:31.461355 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:37:31.461378 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:37:31.461472 5897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:31.458968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35427 (local address 127.1.208.1:51257)
0808 00:37:31.459148 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:31.459157 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:31.459210 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:31.459344 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:31.459344 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:31.459347 (+ 3us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:31.459396 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.459397 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.459967 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.459967 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.460707 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:31.460710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:31.461093 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:31.461094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:31.461151 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:31.461156 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:31.461313 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:31.461314 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:31.461354 (+ 40us) client_negotiation.cc:240] Negotiation successful
0808 00:37:31.461400 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":72,"threads_started":1}
W20260808 00:37:31.461787 5812 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 00:37:32.986073 4865 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:32.986196 4865 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:32.987049 4865 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e669983ed7874e5b9ae1d88a1dc23753: Sending DeleteTablet for 3 replicas of tablet f88b2eae176c4057a784ff6d350f6190
I20260808 00:37:32.987347 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet f88b2eae176c4057a784ff6d350f6190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:32 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:43686
I20260808 00:37:32.987361 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet f88b2eae176c4057a784ff6d350f6190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:32 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:53880
I20260808 00:37:32.987543 5325 tablet_service.cc:1514] Processing DeleteTablet for tablet f88b2eae176c4057a784ff6d350f6190 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:32 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33988
I20260808 00:37:32.987548 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4996
I20260808 00:37:32.994184 5941 tablet_replica.cc:330] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: stopping tablet replica
I20260808 00:37:32.994267 5941 raft_consensus.cc:2229] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:32.994319 5941 raft_consensus.cc:2258] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:32.994561 5941 ts_tablet_manager.cc:1905] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:32.996249 5941 ts_tablet_manager.cc:1918] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:37:32.996312 5941 log.cc:1196] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/f88b2eae176c4057a784ff6d350f6190
I20260808 00:37:32.996510 5941 ts_tablet_manager.cc:1939] T f88b2eae176c4057a784ff6d350f6190 P 42e1b25066764f9a9c199bcdda877066: Deleting consensus metadata
I20260808 00:37:32.996825 4832 catalog_manager.cc:4841] TS 42e1b25066764f9a9c199bcdda877066 (127.1.208.3:41829): tablet f88b2eae176c4057a784ff6d350f6190 (table test-table [id=3633fc99dc18423686ad8916b4691bbe]) successfully deleted
I20260808 00:37:33.001425 5943 tablet_replica.cc:330] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: stopping tablet replica
I20260808 00:37:33.001506 5943 raft_consensus.cc:2229] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:33.001621 5943 raft_consensus.cc:2258] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:33.001894 5943 ts_tablet_manager.cc:1905] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:33.003512 5943 ts_tablet_manager.cc:1918] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 00:37:33.003573 5943 log.cc:1196] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/f88b2eae176c4057a784ff6d350f6190
I20260808 00:37:33.003770 5943 ts_tablet_manager.cc:1939] T f88b2eae176c4057a784ff6d350f6190 P 42fe980ab4774997b4a67e6c95b3927e: Deleting consensus metadata
I20260808 00:37:33.004276 4829 catalog_manager.cc:4841] TS 42fe980ab4774997b4a67e6c95b3927e (127.1.208.2:35427): tablet f88b2eae176c4057a784ff6d350f6190 (table test-table [id=3633fc99dc18423686ad8916b4691bbe]) successfully deleted
W20260808 00:37:33.004877 4831 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:37067 (error 108)
I20260808 00:37:33.005177 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5218
I20260808 00:37:33.005792 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:33.005238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37067 (local address 127.0.0.1:45180)
0808 00:37:33.005712 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:33.005757 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:37067: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":84,"threads_started":1}
W20260808 00:37:33.005882 4831 catalog_manager.cc:4584] TS 6c672a34b6e94d5aa1212e68d2f165f2 (127.1.208.1:37067): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f88b2eae176c4057a784ff6d350f6190: Network error: Client connection negotiation failed: client connection to 127.1.208.1:37067: connect: Connection refused (error 111)
I20260808 00:37:33.018967 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5459
I20260808 00:37:33.029233 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4804
2026-08-08T00:37:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5769 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:33 dist-test-slave-gjw7 krb5kdc[5950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:33 dist-test-slave-gjw7 krb5kdc[5950](info): set up 2 sockets
Aug 08 00:37:33 dist-test-slave-gjw7 krb5kdc[5950](info): commencing operation
krb5kdc: starting...
W20260808 00:37:34.212021 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:34.336669 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:41759
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:41759
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:34.421572 5968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:34.421797 5968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:34.421820 5968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:34.423000 5968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:34.423045 5968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:34.423058 5968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:34.423071 5968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:34.423095 5968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:34.424638 5968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:41759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:41759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:34.424889 5968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:34.425110 5968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:34.430442 5979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.430497 5982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.430474 5968 server_base.cc:1034] running on GCE node
W20260808 00:37:34.430440 5986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.430898 5968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:34.431160 5968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:34.432287 5968 hybrid_clock.cc:648] HybridClock initialized: now 1786149454432275 us; error 29 us; skew 500 ppm
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:34.434737 5968 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:34.435190 5968 webserver.cc:460] Webserver started at http://127.1.208.62:43703/ using document root <none> and password file <none>
I20260808 00:37:34.435406 5968 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:34.435451 5968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:34.435582 5968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:34.436746 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "9fdc668bf6e14be7a03ac919896eb219"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "d67efb0b2b2d10fe4b9b1617256d5809"
server_key_iv: "fe08d67848639d305c653f21e7a073a8"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.437160 5968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "9fdc668bf6e14be7a03ac919896eb219"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "d67efb0b2b2d10fe4b9b1617256d5809"
server_key_iv: "fe08d67848639d305c653f21e7a073a8"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.438517 5968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:34.439213 5996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.439400 5968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:34.439498 5968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "9fdc668bf6e14be7a03ac919896eb219"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "d67efb0b2b2d10fe4b9b1617256d5809"
server_key_iv: "fe08d67848639d305c653f21e7a073a8"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.439560 5968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:34.459901 5968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:34.461992 5968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:34.462141 5968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:34.466439 5968 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:41759
I20260808 00:37:34.466491 6081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:41759 every 8 connection(s)
I20260808 00:37:34.466851 5968 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:34.467799 6082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:34.470278 6082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219: Bootstrap starting.
I20260808 00:37:34.470865 6082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:34.471149 6082 log.cc:824] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:34.471832 6082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219: No bootstrap required, opened a new log
I20260808 00:37:34.472451 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5968
I20260808 00:37:34.472537 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:34.472718 1856 external_mini_cluster.cc:1428] Setting key fc54d12101073ad461b13c3d0f477223
I20260808 00:37:34.473912 6082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } }
I20260808 00:37:34.474090 6082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:34.474128 6082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fdc668bf6e14be7a03ac919896eb219, State: Initialized, Role: FOLLOWER
I20260808 00:37:34.474300 6082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } }
I20260808 00:37:34.474383 6082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:34.474406 6082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:34.474438 6082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:34.475482 6082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } }
I20260808 00:37:34.475593 6082 leader_election.cc:302] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fdc668bf6e14be7a03ac919896eb219; no voters:
I20260808 00:37:34.475759 6082 leader_election.cc:288] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:34.475814 6088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:34.476032 6088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [term 1 LEADER]: Becoming Leader. State: Replica: 9fdc668bf6e14be7a03ac919896eb219, State: Running, Role: LEADER
I20260808 00:37:34.476019 6082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:34.476202 6088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } }
I20260808 00:37:34.476907 6094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fdc668bf6e14be7a03ac919896eb219. Latest consensus state: current_term: 1 leader_uuid: "9fdc668bf6e14be7a03ac919896eb219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } } }
I20260808 00:37:34.477097 6094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:34.477475 6092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fdc668bf6e14be7a03ac919896eb219" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fdc668bf6e14be7a03ac919896eb219" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41759 } } }
I20260808 00:37:34.477557 6092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:34.477649 6112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:34.478189 6112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:34.479866 6112 catalog_manager.cc:1347] Generated new cluster ID: 69b67a0426314ea2b93544e2ffd66f03
I20260808 00:37:34.479926 6112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:34.485523 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.473254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41692 (local address 127.1.208.62:41759)
0808 00:37:34.473500 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:34.473510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:34.473548 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:37:34.473875 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:34.473884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:34.473950 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:34.474101 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:34.474862 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.476223 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.477334 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.477942 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.478055 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.480906 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:34.480911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:34.480912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:34.480926 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:34.482940 (+ 2014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.483367 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.483371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.483371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.483502 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.483855 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.483857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.483857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.484332 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:34.484490 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:34.484940 (+ 450us) server_negotiation.cc:299] Negotiation successful
0808 00:37:34.485177 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":110,"threads_started":1}
I20260808 00:37:34.507680 6112 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:34.508301 6112 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:34.513531 6112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fdc668bf6e14be7a03ac919896eb219: Generated new TSK 0
I20260808 00:37:34.513733 6112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:34.541674 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:34.625552 6124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:34.625833 6124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:34.625866 6124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:34.627641 6124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:34.627696 6124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:34.627724 6124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:34.629151 6124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:34.629397 6124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:34.629587 6124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:34.634333 6137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.634330 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.634371 6138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.634521 6124 server_base.cc:1034] running on GCE node
I20260808 00:37:34.634801 6124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:34.635028 6124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:34.636168 6124 hybrid_clock.cc:648] HybridClock initialized: now 1786149454636153 us; error 28 us; skew 500 ppm
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:34.638581 6124 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:34.638985 6124 webserver.cc:460] Webserver started at http://127.1.208.1:34869/ using document root <none> and password file <none>
I20260808 00:37:34.639187 6124 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:34.639235 6124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:34.639361 6124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:34.640376 6124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "8834c47aebcc42a0b49507153eadeb87"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "542445c1d7cd89da00b23baad4294bfb"
server_key_iv: "90fba3314c865fcdec7839df6ef70d7a"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.640727 6124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "8834c47aebcc42a0b49507153eadeb87"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "542445c1d7cd89da00b23baad4294bfb"
server_key_iv: "90fba3314c865fcdec7839df6ef70d7a"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.642052 6124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:34.642764 6152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.642904 6124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:34.643000 6124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "8834c47aebcc42a0b49507153eadeb87"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "542445c1d7cd89da00b23baad4294bfb"
server_key_iv: "90fba3314c865fcdec7839df6ef70d7a"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.643060 6124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:34.654522 6124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:34.656232 6124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:34.656365 6124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:34.656574 6124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:34.656876 6124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:34.656921 6124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.656951 6124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:34.656975 6124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.662827 6124 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39793
I20260808 00:37:34.662894 6321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39793 every 8 connection(s)
I20260808 00:37:34.663177 6124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:34.667145 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6124
I20260808 00:37:34.667243 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:34.667423 1856 external_mini_cluster.cc:1428] Setting key 7e0e6febfde7a3f02a981180fe0361d1
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:34.672502 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.663848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:48597 (local address 127.1.208.62:41759)
0808 00:37:34.663915 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:34.663917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:34.664109 (+ 192us) server_negotiation.cc:407] Connection header received
0808 00:37:34.664584 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:34.664585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:34.664597 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:34.664651 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:34.665614 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.666117 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.667055 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.667362 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.667429 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.669659 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:34.669661 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:34.669661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:34.669672 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:34.671086 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.671512 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.671513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.671513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.671562 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.671795 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.671796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.671796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.671893 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:34.671937 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:34.672305 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 00:37:34.672430 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:37:34.672669 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.663812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41759 (local address 127.1.208.1:48597)
0808 00:37:34.663981 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:34.664023 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:34.664507 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:34.664708 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:34.664711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:34.665281 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:34.665540 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.665546 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.666182 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.666182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.666977 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.666979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.667442 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.667443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.667579 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.668099 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:34.668107 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:34.669509 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:34.671166 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.671174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.671176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.671448 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.671610 (+ 162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.671611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.671611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.671740 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.671942 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:34.671949 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:34.672133 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 00:37:34.672254 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 00:37:34.672441 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260808 00:37:34.673146 6322 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41759
I20260808 00:37:34.673241 6322 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:34.673444 6322 heartbeater.cc:511] Master 127.1.208.62:41759 requested a full tablet report, sending...
I20260808 00:37:34.673992 6023 ts_manager.cc:194] Registered new tserver with Master: 8834c47aebcc42a0b49507153eadeb87 (127.1.208.1:39793)
I20260808 00:37:34.674582 6023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:48597
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:34.692978 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:34.775866 6329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:34.776080 6329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:34.776103 6329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:34.777330 6329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:34.777374 6329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:34.777400 6329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:34.778846 6329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:34.779062 6329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:34.779269 6329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:37:34.784150 6329 server_base.cc:1034] running on GCE node
W20260808 00:37:34.784122 6338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.784143 6337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.784119 6341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.784564 6329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:34.784791 6329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:34.785938 6329 hybrid_clock.cc:648] HybridClock initialized: now 1786149454785922 us; error 29 us; skew 500 ppm
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:34.788354 6329 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:34.788795 6329 webserver.cc:460] Webserver started at http://127.1.208.2:43775/ using document root <none> and password file <none>
I20260808 00:37:34.789000 6329 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:34.789049 6329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:34.789176 6329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:34.790418 6329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "e6a3ea19033a48c5aa16ae645145f6d3"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "0fd4e7954f6862e0a2dd2119c8dcd0e1"
server_key_iv: "b4a4b1357003fc78168cddca3843d1e7"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.790776 6329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "e6a3ea19033a48c5aa16ae645145f6d3"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "0fd4e7954f6862e0a2dd2119c8dcd0e1"
server_key_iv: "b4a4b1357003fc78168cddca3843d1e7"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.792116 6329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:34.792794 6349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.792956 6329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:34.793041 6329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "e6a3ea19033a48c5aa16ae645145f6d3"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "0fd4e7954f6862e0a2dd2119c8dcd0e1"
server_key_iv: "b4a4b1357003fc78168cddca3843d1e7"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.793110 6329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:34.802016 6329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:34.804172 6329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:34.804317 6329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:34.804540 6329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:34.804843 6329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:34.804877 6329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.804913 6329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:34.804935 6329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.810715 6329 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:36571
I20260808 00:37:34.810791 6519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:36571 every 8 connection(s)
I20260808 00:37:34.811090 6329 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:34.818078 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6329
I20260808 00:37:34.818161 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:34.818348 1856 external_mini_cluster.cc:1428] Setting key 25fecdbf654248ca88f70b33e2f6facb
I20260808 00:37:34.820971 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.811816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:46607 (local address 127.1.208.62:41759)
0808 00:37:34.811880 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:34.811882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:34.812121 (+ 239us) server_negotiation.cc:407] Connection header received
0808 00:37:34.812657 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:34.812659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:34.812673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:34.812726 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:34.813738 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.814254 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.815668 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.815878 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.815952 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.818138 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:34.818140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:34.818140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:34.818149 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:34.819669 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.820047 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.820048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.820049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.820116 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.820341 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.820342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.820342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.820470 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:34.820534 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:34.820780 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 00:37:34.820874 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:34.821177 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.811784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41759 (local address 127.1.208.2:46607)
0808 00:37:34.811978 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:34.812016 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:34.812549 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:34.812798 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:34.812802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:34.813422 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:34.813636 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.813642 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.814344 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.814345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.815585 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.815588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.815960 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.815961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.816103 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.816619 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:34.816628 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:34.818039 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:34.819748 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.819756 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.819758 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.819980 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.820181 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.820182 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.820182 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.820283 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.820529 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:34.820533 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:34.820654 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 00:37:34.820753 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:37:34.820939 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":98,"threads_started":1}
I20260808 00:37:34.821687 6523 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41759
I20260808 00:37:34.821786 6523 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:34.821977 6523 heartbeater.cc:511] Master 127.1.208.62:41759 requested a full tablet report, sending...
I20260808 00:37:34.822362 6023 ts_manager.cc:194] Registered new tserver with Master: e6a3ea19033a48c5aa16ae645145f6d3 (127.1.208.2:36571)
I20260808 00:37:34.822819 6023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:46607
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:34.843837 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:34.926654 6540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:34.926853 6540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:34.926874 6540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:34.928073 6540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:34.928115 6540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:34.928164 6540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:34.929668 6540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41759
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:34.929913 6540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:34.930136 6540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:34.934901 6549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.934994 6540 server_base.cc:1034] running on GCE node
W20260808 00:37:34.934906 6546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:34.935086 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:34.935312 6540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:34.935514 6540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:34.936637 6540 hybrid_clock.cc:648] HybridClock initialized: now 1786149454936627 us; error 27 us; skew 500 ppm
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:34.939064 6540 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:34.939477 6540 webserver.cc:460] Webserver started at http://127.1.208.3:41505/ using document root <none> and password file <none>
I20260808 00:37:34.939688 6540 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:34.939731 6540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:34.939854 6540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:34.940857 6540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "ca5ad87011cf4a1d97b7926e6b809dc8"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "dbf9d1a8a5d678d83f8cc4cd38f329fa"
server_key_iv: "55a622cd00384bc5d4cb8fe166d516ac"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.941183 6540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "ca5ad87011cf4a1d97b7926e6b809dc8"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "dbf9d1a8a5d678d83f8cc4cd38f329fa"
server_key_iv: "55a622cd00384bc5d4cb8fe166d516ac"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.942438 6540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:34.943104 6561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.943262 6540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:34.943351 6540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "ca5ad87011cf4a1d97b7926e6b809dc8"
format_stamp: "Formatted at 2026-08-08 00:37:34 on dist-test-slave-gjw7"
server_key: "dbf9d1a8a5d678d83f8cc4cd38f329fa"
server_key_iv: "55a622cd00384bc5d4cb8fe166d516ac"
server_key_version: "encryptionkey@0"
I20260808 00:37:34.943420 6540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:34.972972 6540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:34.975646 6540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:34.975804 6540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:34.976111 6540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:34.976488 6540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:34.976533 6540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.976574 6540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:34.976593 6540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:34.984157 6540 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:42599
I20260808 00:37:34.984195 6737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:42599 every 8 connection(s)
I20260808 00:37:34.984593 6540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:34.989342 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6540
I20260808 00:37:34.989427 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:34.989616 1856 external_mini_cluster.cc:1428] Setting key f1d3fb828ffc52f215a6eee712d903d0
Aug 08 00:37:34 dist-test-slave-gjw7 krb5kdc[5950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149454, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:34.995033 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.985319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55411 (local address 127.1.208.62:41759)
0808 00:37:34.985378 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:34.985381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:34.985613 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:37:34.986082 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:34.986083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:34.986095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:34.986146 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:34.987397 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.987896 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.989471 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.989751 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.989818 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.992296 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:34.992297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:34.992298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:34.992304 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:34.993615 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.994081 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.994081 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.994082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.994129 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:34.994372 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:34.994372 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:34.994373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:34.994483 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:34.994531 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:34.994876 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 00:37:34.994961 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:34.995235 6744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:34.985311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41759 (local address 127.1.208.3:55411)
0808 00:37:34.985490 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:34.985530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:34.986008 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:34.986270 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:34.986275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:34.987031 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:34.987313 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.987321 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.987977 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.987978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.989334 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:34.989338 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:34.989846 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:34.989847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:34.990012 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:34.990548 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:34.990559 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:34.992210 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:34.993691 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.993699 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.993702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.994011 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.994188 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:34.994189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:34.994189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:34.994319 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:34.994535 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:34.994540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:34.994704 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 00:37:34.994817 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:37:34.994992 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260808 00:37:34.995863 6738 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41759
I20260808 00:37:34.995983 6738 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:34.996188 6738 heartbeater.cc:511] Master 127.1.208.62:41759 requested a full tablet report, sending...
I20260808 00:37:34.996591 6023 ts_manager.cc:194] Registered new tserver with Master: ca5ad87011cf4a1d97b7926e6b809dc8 (127.1.208.3:42599)
I20260808 00:37:34.997076 6023 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:55411
I20260808 00:37:35.001379 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:37:35.015645 6087 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:35.014601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41706 (local address 127.1.208.62:41759)
0808 00:37:35.014667 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:35.014669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:35.014760 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:37:35.014826 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:35.014827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:35.014837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:35.014885 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:35.015568 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41706: BlockingRecv error: recv got EOF from 127.0.0.1:41706 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:37:35.016980 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6124
I20260808 00:37:35.048461 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6329
I20260808 00:37:35.058784 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6540
I20260808 00:37:35.074596 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5968
2026-08-08T00:37:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2045 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:35 dist-test-slave-gjw7 krb5kdc[6756](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:35 dist-test-slave-gjw7 krb5kdc[6756](info): set up 2 sockets
Aug 08 00:37:35 dist-test-slave-gjw7 krb5kdc[6756](info): commencing operation
krb5kdc: starting...
W20260808 00:37:36.278735 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:36.412133 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42519
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42519
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:36.500535 6777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:36.500782 6777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:36.500805 6777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:36.502133 6777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:36.502193 6777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:36.502207 6777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:36.502221 6777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:36.502233 6777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:36.504367 6777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:36.504683 6777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:36.504925 6777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:36.510082 6785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:36.510089 6786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:36.510092 6789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:36.510315 6777 server_base.cc:1034] running on GCE node
I20260808 00:37:36.510653 6777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:36.510996 6777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:36.512146 6777 hybrid_clock.cc:648] HybridClock initialized: now 1786149456512131 us; error 42 us; skew 500 ppm
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6756](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149456, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:36.519280 6777 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:36.519830 6777 webserver.cc:460] Webserver started at http://127.1.208.62:34375/ using document root <none> and password file <none>
I20260808 00:37:36.520082 6777 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:36.520139 6777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:36.520278 6777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:36.521552 6777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "a267f07dcb4048f596373ae64c0f23ba"
format_stamp: "Formatted at 2026-08-08 00:37:36 on dist-test-slave-gjw7"
server_key: "a8b7e37edfc11587fee10814dee4928c"
server_key_iv: "0e94e4534cc5d42ccf2b9eacb14a2d15"
server_key_version: "encryptionkey@0"
I20260808 00:37:36.521927 6777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "a267f07dcb4048f596373ae64c0f23ba"
format_stamp: "Formatted at 2026-08-08 00:37:36 on dist-test-slave-gjw7"
server_key: "a8b7e37edfc11587fee10814dee4928c"
server_key_iv: "0e94e4534cc5d42ccf2b9eacb14a2d15"
server_key_version: "encryptionkey@0"
I20260808 00:37:36.523332 6777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:36.524106 6804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:36.524251 6777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:36.524348 6777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "a267f07dcb4048f596373ae64c0f23ba"
format_stamp: "Formatted at 2026-08-08 00:37:36 on dist-test-slave-gjw7"
server_key: "a8b7e37edfc11587fee10814dee4928c"
server_key_iv: "0e94e4534cc5d42ccf2b9eacb14a2d15"
server_key_version: "encryptionkey@0"
I20260808 00:37:36.524452 6777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:36.550411 6777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:36.552618 6777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:36.552775 6777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:36.556993 6777 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:42519
I20260808 00:37:36.557031 6918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:42519 every 8 connection(s)
I20260808 00:37:36.557459 6777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:36.558435 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6777
I20260808 00:37:36.558452 6920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:36.558537 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:36.558712 1856 external_mini_cluster.cc:1428] Setting key 829dc954f5eb3fadd4cb223ef4ceb8a6
I20260808 00:37:36.562043 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba: Bootstrap starting.
I20260808 00:37:36.562798 6920 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:36.563091 6920 log.cc:824] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:36.563913 6920 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba: No bootstrap required, opened a new log
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6756](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:36.566064 6920 raft_consensus.cc:348] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } }
I20260808 00:37:36.566280 6920 raft_consensus.cc:374] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:36.566326 6920 raft_consensus.cc:729] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a267f07dcb4048f596373ae64c0f23ba, State: Initialized, Role: FOLLOWER
I20260808 00:37:36.566504 6920 consensus_queue.cc:260] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } }
I20260808 00:37:36.566576 6920 raft_consensus.cc:388] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:36.566606 6920 raft_consensus.cc:482] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:36.566659 6920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:36.567283 6920 raft_consensus.cc:504] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } }
I20260808 00:37:36.567387 6920 leader_election.cc:302] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a267f07dcb4048f596373ae64c0f23ba; no voters:
I20260808 00:37:36.567570 6920 leader_election.cc:288] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:36.567616 6928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:36.567736 6928 raft_consensus.cc:686] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [term 1 LEADER]: Becoming Leader. State: Replica: a267f07dcb4048f596373ae64c0f23ba, State: Running, Role: LEADER
I20260808 00:37:36.567878 6920 sys_catalog.cc:564] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:36.567885 6928 consensus_queue.cc:237] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } }
I20260808 00:37:36.568343 6928 sys_catalog.cc:455] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader a267f07dcb4048f596373ae64c0f23ba. Latest consensus state: current_term: 1 leader_uuid: "a267f07dcb4048f596373ae64c0f23ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } } }
I20260808 00:37:36.568498 6928 sys_catalog.cc:458] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:36.568794 6940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:36.569249 6931 sys_catalog.cc:455] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a267f07dcb4048f596373ae64c0f23ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a267f07dcb4048f596373ae64c0f23ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42519 } } }
I20260808 00:37:36.569358 6931 sys_catalog.cc:458] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:36.569650 6940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:36.570271 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:36.559274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.1.208.62:42519)
0808 00:37:36.559482 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:36.559493 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:36.559532 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:37:36.560031 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:36.560043 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:36.560116 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:36.560283 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:36.560922 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:36.562327 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:36.563110 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:36.563623 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:36.563746 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:36.566436 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:36.566442 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:36.566442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:36.566457 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:36.568390 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:36.568872 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:36.568875 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:36.568875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:36.568983 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:36.569212 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:36.569213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:36.569214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:36.569607 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:36.569770 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:36.569905 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 00:37:36.570050 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260808 00:37:36.571583 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:36.571746 6940 catalog_manager.cc:1347] Generated new cluster ID: 05bcc507647a421da835b7020412d13a
I20260808 00:37:36.571782 6940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:36.592289 6940 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:36.593161 6940 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:36.599421 6940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a267f07dcb4048f596373ae64c0f23ba: Generated new TSK 0
I20260808 00:37:36.599642 6940 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:36.600790 6923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:36.597207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49594 (local address 127.1.208.62:42519)
0808 00:37:36.597356 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:36.597358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:36.597431 (+ 73us) server_negotiation.cc:407] Connection header received
0808 00:37:36.597509 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:36.597510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:36.597521 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:36.597569 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:36.597982 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:36.598511 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:36.599323 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:36.599615 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:36.599681 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:36.600352 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:36.600354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:36.600354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:36.600546 (+ 192us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:36.600564 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:36.600570 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:36.600617 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:36.600684 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:37:36.600713 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 00:37:36.603484 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6777
2026-08-08T00:37:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1536 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6969](info): set up 2 sockets
Aug 08 00:37:36 dist-test-slave-gjw7 krb5kdc[6969](info): commencing operation
krb5kdc: starting...
W20260808 00:37:37.806355 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:37 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:37.926396 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37281
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:38.008559 6994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:38.008772 6994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:38.008792 6994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:38.009999 6994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:38.010051 6994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:38.010068 6994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:38.010085 6994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:38.010113 6994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:38.012229 6994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:38.012758 6994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:38.013048 6994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:38.018896 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:38.018891 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:38.019374 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.019757 6994 server_base.cc:1034] running on GCE node
I20260808 00:37:38.020017 6994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:38.020295 6994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:38.021449 6994 hybrid_clock.cc:648] HybridClock initialized: now 1786149458021438 us; error 29 us; skew 500 ppm
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:38.023777 6994 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:38.024211 6994 webserver.cc:460] Webserver started at http://127.1.208.62:42905/ using document root <none> and password file <none>
I20260808 00:37:38.024421 6994 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:38.024468 6994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:38.024596 6994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:38.025720 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "d82d846b752c4a4e98668a0f8316cc2f"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "58001cee6934896458671a0e22e99e05"
server_key_iv: "b766a83d030ecae967a6953be05dae88"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.026070 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "d82d846b752c4a4e98668a0f8316cc2f"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "58001cee6934896458671a0e22e99e05"
server_key_iv: "b766a83d030ecae967a6953be05dae88"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.027338 6994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:38.027988 7018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.028141 6994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:38.028225 6994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "d82d846b752c4a4e98668a0f8316cc2f"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "58001cee6934896458671a0e22e99e05"
server_key_iv: "b766a83d030ecae967a6953be05dae88"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.028286 6994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:38.035195 6994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:38.036862 6994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:38.036986 6994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:38.040938 6994 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37281
I20260808 00:37:38.040984 7101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37281 every 8 connection(s)
I20260808 00:37:38.041321 6994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:38.042416 7103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:38.044644 7103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f: Bootstrap starting.
I20260808 00:37:38.045217 7103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:38.045521 7103 log.cc:824] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:38.046165 7103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f: No bootstrap required, opened a new log
I20260808 00:37:38.047770 7103 raft_consensus.cc:348] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } }
I20260808 00:37:38.047905 7103 raft_consensus.cc:374] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:38.047937 7103 raft_consensus.cc:729] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d82d846b752c4a4e98668a0f8316cc2f, State: Initialized, Role: FOLLOWER
I20260808 00:37:38.048079 7103 consensus_queue.cc:260] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } }
I20260808 00:37:38.048156 7103 raft_consensus.cc:388] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:38.048190 7103 raft_consensus.cc:482] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:38.048237 7103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:38.049012 7103 raft_consensus.cc:504] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } }
I20260808 00:37:38.049104 7103 leader_election.cc:302] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d82d846b752c4a4e98668a0f8316cc2f; no voters:
I20260808 00:37:38.049261 7103 leader_election.cc:288] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:38.049309 7107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:38.049463 7107 raft_consensus.cc:686] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [term 1 LEADER]: Becoming Leader. State: Replica: d82d846b752c4a4e98668a0f8316cc2f, State: Running, Role: LEADER
I20260808 00:37:38.049548 7103 sys_catalog.cc:564] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:38.049552 7107 consensus_queue.cc:237] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } }
I20260808 00:37:38.049932 7108 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d82d846b752c4a4e98668a0f8316cc2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } } }
I20260808 00:37:38.050026 7108 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:38.049940 7109 sys_catalog.cc:455] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader d82d846b752c4a4e98668a0f8316cc2f. Latest consensus state: current_term: 1 leader_uuid: "d82d846b752c4a4e98668a0f8316cc2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d82d846b752c4a4e98668a0f8316cc2f" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37281 } } }
I20260808 00:37:38.050158 7109 sys_catalog.cc:458] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:38.050331 7120 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:38.050850 7120 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:38.051198 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6994
I20260808 00:37:38.051295 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:38.051476 1856 external_mini_cluster.cc:1428] Setting key 722a36c4431ea34e724d302408c3b42f
I20260808 00:37:38.052561 7120 catalog_manager.cc:1347] Generated new cluster ID: f1600e48eb2149c39a076afaa59bfb04
I20260808 00:37:38.052613 7120 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:38.060777 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.051909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36482 (local address 127.1.208.62:37281)
0808 00:37:38.052067 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:38.052077 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:38.052252 (+ 175us) server_negotiation.cc:407] Connection header received
0808 00:37:38.052399 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:38.052414 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:38.052471 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:38.052611 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:38.053354 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.054285 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.055046 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.055459 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.055552 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.057777 (+ 2225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:38.057781 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:38.057781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:38.057794 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:38.059328 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.059653 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.059656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.059657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.059750 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.059929 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.059930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.059930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.060157 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:38.060301 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:38.060399 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:37:38.060559 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260808 00:37:38.065452 7120 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:38.066124 7120 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:38.080422 7120 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d82d846b752c4a4e98668a0f8316cc2f: Generated new TSK 0
I20260808 00:37:38.080655 7120 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:38.108155 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:38.225214 7149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:38.225440 7149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:38.225461 7149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:38.226605 7149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:38.226646 7149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:38.226673 7149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:38.228091 7149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:38.228315 7149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:38.228535 7149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:38.233451 7158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:38.233457 7159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.233507 7149 server_base.cc:1034] running on GCE node
W20260808 00:37:38.233457 7164 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.233815 7149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:38.234050 7149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:38.235230 7149 hybrid_clock.cc:648] HybridClock initialized: now 1786149458235207 us; error 28 us; skew 500 ppm
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:38.237787 7149 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:38.238183 7149 webserver.cc:460] Webserver started at http://127.1.208.1:35957/ using document root <none> and password file <none>
I20260808 00:37:38.238379 7149 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:38.238435 7149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:38.238555 7149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:38.239612 7149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "0670a094c9d44ffeb28b5512491b02da"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "813d2855f1efc86294bfdd38cd6c52ca"
server_key_iv: "24146f931d21513a40548964ddce80ce"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.239952 7149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "0670a094c9d44ffeb28b5512491b02da"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "813d2855f1efc86294bfdd38cd6c52ca"
server_key_iv: "24146f931d21513a40548964ddce80ce"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.241257 7149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:38.242034 7185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.242178 7149 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:38.242264 7149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "0670a094c9d44ffeb28b5512491b02da"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "813d2855f1efc86294bfdd38cd6c52ca"
server_key_iv: "24146f931d21513a40548964ddce80ce"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.242327 7149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:38.253747 7149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:38.255882 7149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:38.256029 7149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:38.256253 7149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:38.256554 7149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:38.256588 7149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.256618 7149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:38.256642 7149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.262282 7149 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:38145
I20260808 00:37:38.262346 7385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:38145 every 8 connection(s)
I20260808 00:37:38.262645 7149 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:38.264057 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7149
I20260808 00:37:38.264154 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:38.264339 1856 external_mini_cluster.cc:1428] Setting key ab17027fdbc5e248be95f712e74678e0
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:38.271934 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.263321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:35859 (local address 127.1.208.62:37281)
0808 00:37:38.263391 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:38.263394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:38.263570 (+ 176us) server_negotiation.cc:407] Connection header received
0808 00:37:38.264091 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:38.264093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:38.264112 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:38.264179 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:38.265167 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.265853 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.266838 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.267051 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.267109 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.269381 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:38.269383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:38.269383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:38.269391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:38.270654 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.271021 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.271022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.271023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.271075 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.271295 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.271295 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.271296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.271418 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:38.271475 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:38.271739 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 00:37:38.271839 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:37:38.272078 7389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.263286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37281 (local address 127.1.208.1:35859)
0808 00:37:38.263448 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:38.263484 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:38.263997 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:38.264272 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:38.264277 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:38.264880 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:38.265072 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.265079 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.265954 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.265955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.266745 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.266747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.267124 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.267124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.267247 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.267734 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:38.267742 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:38.269249 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:38.270730 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.270737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.270739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.270953 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.271129 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.271130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.271130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.271238 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.271478 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:38.271485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:38.271617 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 00:37:38.271706 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:37:38.271858 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
I20260808 00:37:38.272580 7386 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37281
I20260808 00:37:38.272667 7386 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:38.272840 7386 heartbeater.cc:511] Master 127.1.208.62:37281 requested a full tablet report, sending...
I20260808 00:37:38.273445 7049 ts_manager.cc:194] Registered new tserver with Master: 0670a094c9d44ffeb28b5512491b02da (127.1.208.1:38145)
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
I20260808 00:37:38.274042 7049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:35859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:38.290166 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:38.374127 7393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:38.374364 7393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:38.374387 7393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:38.375545 7393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:38.375586 7393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:38.375610 7393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:38.377017 7393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:38.377226 7393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:38.377447 7393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:38.382215 7403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.382333 7393 server_base.cc:1034] running on GCE node
W20260808 00:37:38.382215 7400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:38.382285 7401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.382633 7393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:38.382864 7393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:38.384037 7393 hybrid_clock.cc:648] HybridClock initialized: now 1786149458384022 us; error 28 us; skew 500 ppm
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:38.386595 7393 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:38.387027 7393 webserver.cc:460] Webserver started at http://127.1.208.2:42881/ using document root <none> and password file <none>
I20260808 00:37:38.387220 7393 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:38.387269 7393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:38.387358 7393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:38.388357 7393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "733e1c91367d4306a978559e8714bca9"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "e8ab7ba3085fdb12c6b71a1258642657"
server_key_iv: "47a4c5e3b30172008bb36ddadf5cd3b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.388659 7393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "733e1c91367d4306a978559e8714bca9"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "e8ab7ba3085fdb12c6b71a1258642657"
server_key_iv: "47a4c5e3b30172008bb36ddadf5cd3b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.389900 7393 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:37:38.390594 7415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.390748 7393 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.390831 7393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "733e1c91367d4306a978559e8714bca9"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "e8ab7ba3085fdb12c6b71a1258642657"
server_key_iv: "47a4c5e3b30172008bb36ddadf5cd3b3"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.390894 7393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:38.400523 7393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:38.402266 7393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:38.402397 7393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:38.402601 7393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:38.402884 7393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:38.402918 7393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.402947 7393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:38.402973 7393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.408736 7393 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:44975
I20260808 00:37:38.408775 7630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:44975 every 8 connection(s)
I20260808 00:37:38.409071 7393 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:38.415472 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7393
I20260808 00:37:38.415570 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:38.415751 1856 external_mini_cluster.cc:1428] Setting key c28151892275f138ec9d3038724e0c7d
I20260808 00:37:38.418062 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.409836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:53291 (local address 127.1.208.62:37281)
0808 00:37:38.409888 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:38.409890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:38.410078 (+ 188us) server_negotiation.cc:407] Connection header received
0808 00:37:38.410583 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:38.410584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:38.410595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:38.410654 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:38.411575 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.412075 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.413020 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.413196 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.413239 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.415324 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:38.415325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:38.415326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:38.415333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:38.416731 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.417181 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.417181 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.417182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.417227 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.417489 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.417490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.417490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.417595 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:38.417644 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:38.417934 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 00:37:38.417998 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:38.418390 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.409774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37281 (local address 127.1.208.2:53291)
0808 00:37:38.409954 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:38.409997 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:38.410492 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:38.410730 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:38.410733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:38.411301 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:38.411496 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.411501 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.412141 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.412142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.412959 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.412961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.413249 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.413249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.413365 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.413857 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:38.413866 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:38.415228 (+ 1362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:38.416810 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.416817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.416820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.417114 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.417266 (+ 152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.417267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.417267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.417432 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.417647 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:38.417653 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:38.417808 (+ 155us) client_negotiation.cc:769] Sending connection context
0808 00:37:38.417924 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:37:38.418097 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260808 00:37:38.418861 7631 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37281
I20260808 00:37:38.418948 7631 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:38.419117 7631 heartbeater.cc:511] Master 127.1.208.62:37281 requested a full tablet report, sending...
I20260808 00:37:38.419477 7049 ts_manager.cc:194] Registered new tserver with Master: 733e1c91367d4306a978559e8714bca9 (127.1.208.2:44975)
I20260808 00:37:38.419804 7049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:53291
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:38.442233 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:40551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:38.524515 7641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:38.524720 7641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:38.524744 7641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:38.525854 7641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:38.525892 7641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:38.525916 7641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:38.527280 7641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37281
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:38.527491 7641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:38.527671 7641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:38.532624 7649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.532724 7641 server_base.cc:1034] running on GCE node
W20260808 00:37:38.532647 7648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:38.532658 7651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:38.533072 7641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:38.533345 7641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:38.534493 7641 hybrid_clock.cc:648] HybridClock initialized: now 1786149458534477 us; error 27 us; skew 500 ppm
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:38.536958 7641 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:38.537456 7641 webserver.cc:460] Webserver started at http://127.1.208.3:34731/ using document root <none> and password file <none>
I20260808 00:37:38.537662 7641 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:38.537708 7641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:38.537842 7641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:38.538908 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "11e75f052e2944b2b5b2c6edfc2092cd"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "6b091d9b8fd7c5b77cf484f829c94f79"
server_key_iv: "d5949245bfd38f168d357891c1153307"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.539207 7641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "11e75f052e2944b2b5b2c6edfc2092cd"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "6b091d9b8fd7c5b77cf484f829c94f79"
server_key_iv: "d5949245bfd38f168d357891c1153307"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.540449 7641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:38.541111 7661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.541267 7641 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:38.541388 7641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "11e75f052e2944b2b5b2c6edfc2092cd"
format_stamp: "Formatted at 2026-08-08 00:37:38 on dist-test-slave-gjw7"
server_key: "6b091d9b8fd7c5b77cf484f829c94f79"
server_key_iv: "d5949245bfd38f168d357891c1153307"
server_key_version: "encryptionkey@0"
I20260808 00:37:38.541457 7641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:38.562276 7641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:38.564065 7641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:38.564198 7641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:38.564431 7641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:38.564742 7641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:38.564775 7641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.564805 7641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:38.564828 7641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:38.570645 7641 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:44951
I20260808 00:37:38.570705 7868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:44951 every 8 connection(s)
I20260808 00:37:38.571008 7641 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:38.577459 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7641
I20260808 00:37:38.577539 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:38.577739 1856 external_mini_cluster.cc:1428] Setting key 412337b1a5fdef9d56deaed203e36553
I20260808 00:37:38.579754 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.571679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:50781 (local address 127.1.208.62:37281)
0808 00:37:38.571727 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:38.571729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:38.571935 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:37:38.572431 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:38.572432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:38.572441 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:38.572497 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:38.573462 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.573957 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.574877 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.575055 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.575099 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.577151 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:38.577152 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:38.577153 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:38.577160 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:38.578597 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.579028 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.579029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.579029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.579079 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.579281 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.579282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.579282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.579386 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:38.579434 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:38.579631 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 00:37:38.579690 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:38.579926 7876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.571655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37281 (local address 127.1.208.3:50781)
0808 00:37:38.571816 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:38.571859 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:38.572350 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:38.572570 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:38.572573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:38.573154 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:38.573395 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.573401 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.574024 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.574024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.574816 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:38.574818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.575108 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.575108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:38.575204 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.575677 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:38.575686 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:38.577062 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:38.578659 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.578666 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.578668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.578955 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.579139 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:38.579140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:38.579140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:38.579238 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:38.579424 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:38.579427 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:38.579537 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 00:37:38.579611 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:37:38.579732 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260808 00:37:38.580380 7869 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37281
I20260808 00:37:38.580489 7869 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:38.580668 7869 heartbeater.cc:511] Master 127.1.208.62:37281 requested a full tablet report, sending...
I20260808 00:37:38.581028 7049 ts_manager.cc:194] Registered new tserver with Master: 11e75f052e2944b2b5b2c6edfc2092cd (127.1.208.3:44951)
I20260808 00:37:38.581429 7049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:50781
I20260808 00:37:38.589345 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[6969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149458, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:38.612257 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:38.605990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.1.208.62:37281)
0808 00:37:38.606051 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:38.606054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:38.606152 (+ 98us) server_negotiation.cc:407] Connection header received
0808 00:37:38.606210 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:38.606211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:38.606220 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:38.606274 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:38.606800 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.607294 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.607990 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:38.608164 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:38.608204 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:38.610295 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:38.610297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:38.610297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:38.610304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:38.611383 (+ 1079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.611666 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.611667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.611667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.611709 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:38.611844 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:38.611844 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:38.611844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:38.611985 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:38.612057 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:38.612129 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 00:37:38.612189 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 00:37:38.612497 7049 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36498
I20260808 00:37:38.612998 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7149
I20260808 00:37:38.628300 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7393
I20260808 00:37:38.640403 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7641
I20260808 00:37:38.656764 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6994
2026-08-08T00:37:38Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2042 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[7895](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[7895](info): set up 2 sockets
Aug 08 00:37:38 dist-test-slave-gjw7 krb5kdc[7895](info): commencing operation
krb5kdc: starting...
W20260808 00:37:39.834731 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:39 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:39.956247 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42733
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42733
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:40.039027 7923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:40.039227 7923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:40.039249 7923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:40.040473 7923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:40.040517 7923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:40.040530 7923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:40.040542 7923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:40.040566 7923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:40.042075 7923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7923
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:40.042309 7923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:40.042515 7923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:40.047111 7932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:40.047116 7928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.047361 7923 server_base.cc:1034] running on GCE node
W20260808 00:37:40.047415 7929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.047667 7923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:40.047909 7923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:40.049047 7923 hybrid_clock.cc:648] HybridClock initialized: now 1786149460049029 us; error 34 us; skew 500 ppm
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:40.051473 7923 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:40.051870 7923 webserver.cc:460] Webserver started at http://127.1.208.62:38735/ using document root <none> and password file <none>
I20260808 00:37:40.052038 7923 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:40.052076 7923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:40.052170 7923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:40.053155 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "1a24e818e65444a6810dd5ad64cef255"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "e443e1ae12fcdabb0a3bf02e72431162"
server_key_iv: "d3e9ad4335b3df27c95ab155aa61e70d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.053519 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "1a24e818e65444a6810dd5ad64cef255"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "e443e1ae12fcdabb0a3bf02e72431162"
server_key_iv: "d3e9ad4335b3df27c95ab155aa61e70d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.054708 7923 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:40.055311 7945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.055462 7923 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:40.055545 7923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "1a24e818e65444a6810dd5ad64cef255"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "e443e1ae12fcdabb0a3bf02e72431162"
server_key_iv: "d3e9ad4335b3df27c95ab155aa61e70d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.055600 7923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:40.073071 7923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:40.075358 7923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:40.075487 7923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:40.079310 7923 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:42733
I20260808 00:37:40.079389 8039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:42733 every 8 connection(s)
I20260808 00:37:40.079929 7923 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:40.080592 8040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:40.081445 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7923
I20260808 00:37:40.081544 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:40.081723 1856 external_mini_cluster.cc:1428] Setting key ce69cb8438d6f0912011da0458693b48
I20260808 00:37:40.083432 8040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255: Bootstrap starting.
I20260808 00:37:40.084084 8040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:40.084419 8040 log.cc:824] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:40.085150 8040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255: No bootstrap required, opened a new log
I20260808 00:37:40.086839 8040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } }
I20260808 00:37:40.087023 8040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:40.087065 8040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a24e818e65444a6810dd5ad64cef255, State: Initialized, Role: FOLLOWER
I20260808 00:37:40.087189 8040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } }
I20260808 00:37:40.087262 8040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:40.087297 8040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:40.087344 8040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:40.087894 8040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } }
I20260808 00:37:40.087998 8040 leader_election.cc:302] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a24e818e65444a6810dd5ad64cef255; no voters:
I20260808 00:37:40.088164 8040 leader_election.cc:288] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:40.088207 8046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:40.088361 8046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [term 1 LEADER]: Becoming Leader. State: Replica: 1a24e818e65444a6810dd5ad64cef255, State: Running, Role: LEADER
I20260808 00:37:40.088430 8040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:40.088456 8046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } }
I20260808 00:37:40.088796 8046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a24e818e65444a6810dd5ad64cef255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } } }
I20260808 00:37:40.088904 8046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:40.089048 8046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a24e818e65444a6810dd5ad64cef255. Latest consensus state: current_term: 1 leader_uuid: "1a24e818e65444a6810dd5ad64cef255" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a24e818e65444a6810dd5ad64cef255" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42733 } } }
I20260808 00:37:40.089108 8046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:40.089231 8055 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:40.089761 8055 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:40.093108 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.082217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47676 (local address 127.1.208.62:42733)
0808 00:37:40.082416 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.082425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.082462 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:37:40.083109 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.083119 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.083186 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.083360 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.083947 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.085334 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.086058 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.087290 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.087408 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.090085 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.090089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.090090 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.090101 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.091706 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.092036 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.092039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.092040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.092126 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.092300 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.092301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.092301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.092537 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.092680 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.092784 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.092913 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260808 00:37:40.094127 8055 catalog_manager.cc:1347] Generated new cluster ID: 082f1f9aff224635892c63f470d1ff8b
I20260808 00:37:40.094183 8055 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:40.106493 8055 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:40.107007 8055 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:40.110476 8055 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a24e818e65444a6810dd5ad64cef255: Generated new TSK 0
I20260808 00:37:40.110642 8055 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:40.144512 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:40.227451 8084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:40.227654 8084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:40.227676 8084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:40.228817 8084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:40.228858 8084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:40.228888 8084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:40.230427 8084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:40.230659 8084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:40.230880 8084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:40.235661 8097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.235790 8084 server_base.cc:1034] running on GCE node
W20260808 00:37:40.235669 8095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:40.235702 8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.236127 8084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:40.236379 8084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:40.237519 8084 hybrid_clock.cc:648] HybridClock initialized: now 1786149460237505 us; error 27 us; skew 500 ppm
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:40.239912 8084 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:40.240370 8084 webserver.cc:460] Webserver started at http://127.1.208.1:37701/ using document root <none> and password file <none>
I20260808 00:37:40.240566 8084 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:40.240610 8084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:40.240738 8084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:40.241796 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "6c0b7b10de1c429f91881eff4407db88"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c46055c0bd05210f7c48b3aa99fe1e5f"
server_key_iv: "b82d230f0cc57c60a0c10b14c1c724b2"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.242128 8084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "6c0b7b10de1c429f91881eff4407db88"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c46055c0bd05210f7c48b3aa99fe1e5f"
server_key_iv: "b82d230f0cc57c60a0c10b14c1c724b2"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.243367 8084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:40.244047 8110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.244195 8084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:40.244278 8084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "6c0b7b10de1c429f91881eff4407db88"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c46055c0bd05210f7c48b3aa99fe1e5f"
server_key_iv: "b82d230f0cc57c60a0c10b14c1c724b2"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.244351 8084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:40.258138 8084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:40.260003 8084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:40.260134 8084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:40.260352 8084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:40.260646 8084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:40.260680 8084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.260710 8084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:40.260733 8084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.266305 8084 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39147
I20260808 00:37:40.266376 8305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39147 every 8 connection(s)
I20260808 00:37:40.266652 8084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:40.269552 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8084
I20260808 00:37:40.269660 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:40.269846 1856 external_mini_cluster.cc:1428] Setting key ee4a7fea972f0b2556629980b3d43475
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:40.276238 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.267317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:42581 (local address 127.1.208.62:42733)
0808 00:37:40.267366 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.267368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.267583 (+ 215us) server_negotiation.cc:407] Connection header received
0808 00:37:40.268546 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.268547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.268559 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.268614 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.269246 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.270006 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.271304 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.271510 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.271564 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.273699 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.273700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.273700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.273707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.274901 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.275356 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.275357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.275358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.275408 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.275649 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.275650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.275650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.275745 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.275789 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.276103 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.276173 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:40.276480 8309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.267282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42733 (local address 127.1.208.1:42581)
0808 00:37:40.267459 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:40.267495 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:40.268433 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:40.268696 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:40.268701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:40.268963 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:40.269146 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.269151 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.270099 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.270101 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.271207 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.271210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.271584 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.271585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.271719 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.272187 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:40.272198 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:40.273616 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:40.274986 (+ 1370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.274996 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.274998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.275286 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.275463 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.275463 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.275463 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.275596 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.275794 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:40.275800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:40.275971 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 00:37:40.276085 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 00:37:40.276254 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260808 00:37:40.277027 8306 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42733
I20260808 00:37:40.277128 8306 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:40.277338 8306 heartbeater.cc:511] Master 127.1.208.62:42733 requested a full tablet report, sending...
I20260808 00:37:40.277947 7978 ts_manager.cc:194] Registered new tserver with Master: 6c0b7b10de1c429f91881eff4407db88 (127.1.208.1:39147)
I20260808 00:37:40.278860 7978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:42581
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:40.297242 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:40.379866 8318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:40.380071 8318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:40.380095 8318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:40.381238 8318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:40.381282 8318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:40.381350 8318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:40.382870 8318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:40.383092 8318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:40.383306 8318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:40.388150 8326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.388363 8318 server_base.cc:1034] running on GCE node
W20260808 00:37:40.388152 8327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:40.388228 8329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.388737 8318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:40.388958 8318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:40.390130 8318 hybrid_clock.cc:648] HybridClock initialized: now 1786149460390114 us; error 30 us; skew 500 ppm
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:40.392613 8318 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:40.393033 8318 webserver.cc:460] Webserver started at http://127.1.208.2:37659/ using document root <none> and password file <none>
I20260808 00:37:40.393229 8318 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:40.393275 8318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:40.393435 8318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:40.394450 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "8c2352c733b1476fbbd83774f852a064"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c56120ac3d2538bd6c576e5221e54e0c"
server_key_iv: "0b12e69aa9110b983cba54ee89ee188d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.394789 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "8c2352c733b1476fbbd83774f852a064"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c56120ac3d2538bd6c576e5221e54e0c"
server_key_iv: "0b12e69aa9110b983cba54ee89ee188d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.396044 8318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:37:40.396713 8336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.396863 8318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:40.396948 8318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "8c2352c733b1476fbbd83774f852a064"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "c56120ac3d2538bd6c576e5221e54e0c"
server_key_iv: "0b12e69aa9110b983cba54ee89ee188d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.397014 8318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:40.403888 8318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:40.405731 8318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:40.405864 8318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:40.406069 8318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:40.406360 8318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:40.406394 8318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.406421 8318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:40.406438 8318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.411999 8318 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:45787
I20260808 00:37:40.412078 8532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:45787 every 8 connection(s)
I20260808 00:37:40.412335 8318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:40.420840 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.413010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:58915 (local address 127.1.208.62:42733)
0808 00:37:40.413054 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.413056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.413247 (+ 191us) server_negotiation.cc:407] Connection header received
0808 00:37:40.414212 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.414213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.414226 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.414277 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.414891 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.415386 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.416357 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.416541 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.416595 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.418585 (+ 1990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.418587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.418587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.418595 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.419641 (+ 1046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.419992 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.419994 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.419994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.420048 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.420255 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.420256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.420256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.420362 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.420420 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.420670 (+ 250us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.420749 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:40.420991 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.412971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42733 (local address 127.1.208.2:58915)
0808 00:37:40.413140 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:40.413172 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:40.414105 (+ 933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:40.414348 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:40.414352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:40.414614 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:40.414793 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.414799 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.415470 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.415472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.416272 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.416275 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.416610 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.416611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.416727 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.417151 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:40.417160 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:40.418496 (+ 1336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:40.419713 (+ 1217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.419720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.419722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.419926 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.420101 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.420102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.420102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.420199 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.420419 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:40.420426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:40.420553 (+ 127us) client_negotiation.cc:769] Sending connection context
0808 00:37:40.420640 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:37:40.420786 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":96,"threads_started":1}
I20260808 00:37:40.421420 8534 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42733
I20260808 00:37:40.421506 8534 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:40.421645 8534 heartbeater.cc:511] Master 127.1.208.62:42733 requested a full tablet report, sending...
I20260808 00:37:40.421972 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8318
I20260808 00:37:40.422019 7978 ts_manager.cc:194] Registered new tserver with Master: 8c2352c733b1476fbbd83774f852a064 (127.1.208.2:45787)
I20260808 00:37:40.422083 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:40.422261 1856 external_mini_cluster.cc:1428] Setting key ef4b0a86170f1297467d44780bcf6426
I20260808 00:37:40.422411 7978 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:58915
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:40.448261 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:40.530190 8540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:40.530388 8540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:40.530411 8540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:40.531520 8540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:40.531559 8540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:40.531584 8540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:40.533097 8540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:40.533349 8540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:40.533556 8540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:40.538522 8556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.538522 8540 server_base.cc:1034] running on GCE node
W20260808 00:37:40.538473 8549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:40.538506 8550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:40.538885 8540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:40.539105 8540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:40.540246 8540 hybrid_clock.cc:648] HybridClock initialized: now 1786149460540221 us; error 37 us; skew 500 ppm
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:40.542726 8540 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:40.543156 8540 webserver.cc:460] Webserver started at http://127.1.208.3:38419/ using document root <none> and password file <none>
I20260808 00:37:40.543352 8540 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:40.543401 8540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:40.543524 8540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:40.544524 8540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "1712ce0ab5074d2a820291355943f40e"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "09da50fc0a040d7ca72f76f9f85e52ef"
server_key_iv: "726b7702917f27c2594ff584699ea97d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.544854 8540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "1712ce0ab5074d2a820291355943f40e"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "09da50fc0a040d7ca72f76f9f85e52ef"
server_key_iv: "726b7702917f27c2594ff584699ea97d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.546237 8540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:40.546912 8568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.547070 8540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:40.547156 8540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "1712ce0ab5074d2a820291355943f40e"
format_stamp: "Formatted at 2026-08-08 00:37:40 on dist-test-slave-gjw7"
server_key: "09da50fc0a040d7ca72f76f9f85e52ef"
server_key_iv: "726b7702917f27c2594ff584699ea97d"
server_key_version: "encryptionkey@0"
I20260808 00:37:40.547219 8540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:40.565231 8540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:40.567106 8540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:40.567216 8540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:40.567445 8540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:40.567744 8540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:40.567776 8540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.567798 8540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:40.567821 8540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:40.573827 8540 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:43719
I20260808 00:37:40.573889 8758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:43719 every 8 connection(s)
I20260808 00:37:40.574172 8540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:40.582846 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.574872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:34463 (local address 127.1.208.62:42733)
0808 00:37:40.574931 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.574933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.575136 (+ 203us) server_negotiation.cc:407] Connection header received
0808 00:37:40.576106 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.576107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.576118 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.576175 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.576767 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.577268 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.578246 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.578442 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.578488 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.580442 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.580443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.580443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.580449 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.581698 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.582041 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.582042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.582042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.582090 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.582347 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.582347 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.582347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.582452 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.582504 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.582711 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.582774 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:40.582978 8762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.574834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42733 (local address 127.1.208.3:34463)
0808 00:37:40.575006 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:40.575056 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:40.576013 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:40.576250 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:40.576253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:40.576503 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:40.576690 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.576694 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.577378 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.577379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.578164 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:40.578166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.578499 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.578499 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:40.578595 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.579015 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:40.579023 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:40.580360 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:40.581770 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.581776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.581778 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.581985 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.582140 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:40.582141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:40.582141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:40.582293 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:40.582500 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:40.582503 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:40.582610 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 00:37:40.582689 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:37:40.582806 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260808 00:37:40.583150 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8540
I20260808 00:37:40.583245 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:40.583459 1856 external_mini_cluster.cc:1428] Setting key 23f07ad6202e27568d055cd3d27478c5
I20260808 00:37:40.583444 8759 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42733
I20260808 00:37:40.583526 8759 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:40.583705 8759 heartbeater.cc:511] Master 127.1.208.62:42733 requested a full tablet report, sending...
I20260808 00:37:40.584115 7976 ts_manager.cc:194] Registered new tserver with Master: 1712ce0ab5074d2a820291355943f40e (127.1.208.3:43719)
I20260808 00:37:40.584535 7976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:34463
I20260808 00:37:40.595013 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:40.617947 8777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.610918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40206 (local address 127.1.208.1:39147)
0808 00:37:40.611065 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.611070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.611081 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:40.611106 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.611108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.611122 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.611193 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.611685 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.612396 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.613089 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.613476 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.613531 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.615562 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.615571 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.615571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.615578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.616870 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.617198 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.617202 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.617202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.617337 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.617497 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.617498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.617498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.617636 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.617745 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.617781 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.617847 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[7895](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:40.625138 8043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:40.619148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47684 (local address 127.1.208.62:42733)
0808 00:37:40.619214 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:40.619215 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:40.619238 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:37:40.619261 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:40.619262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:40.619271 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:40.619313 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:40.619737 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.620226 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.620857 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:40.621058 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:40.621112 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:40.623172 (+ 2060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:40.623173 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:40.623173 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:40.623180 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:40.624250 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.624538 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.624539 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.624539 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.624586 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:40.624712 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:40.624712 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:40.624712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:40.624842 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:40.624915 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:40.624990 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:37:40.625054 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 00:37:40.625428 7976 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47684
I20260808 00:37:40.625746 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8084
I20260808 00:37:40.635895 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8318
I20260808 00:37:40.667982 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8540
I20260808 00:37:40.678581 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7923
2026-08-08T00:37:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2019 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[8785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[8785](info): set up 2 sockets
Aug 08 00:37:40 dist-test-slave-gjw7 krb5kdc[8785](info): commencing operation
krb5kdc: starting...
W20260808 00:37:41.856046 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:41 dist-test-slave-gjw7 krb5kdc[8785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:41.976365 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43757
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:43757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:42.060643 8812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:42.060839 8812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:42.060860 8812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:42.061966 8812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:42.062007 8812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:42.062021 8812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:42.062031 8812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:42.062047 8812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:42.063500 8812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:43757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:42.063695 8812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:42.063860 8812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:37:42.068485 8812 server_base.cc:1034] running on GCE node
W20260808 00:37:42.068455 8821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.068456 8819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.068486 8818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.068941 8812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:42.069228 8812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:42.070369 8812 hybrid_clock.cc:648] HybridClock initialized: now 1786149462070359 us; error 48 us; skew 500 ppm
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:42.077035 8812 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:42.077395 8812 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:42.077453 8812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:42.077579 8812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:42.078616 8812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "6341379a5aaf4c0b951fbc91211605a5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "81a0366e99bd17845ca989d00b63e0c9"
server_key_iv: "33125b3a14f081ce02a7da80dde7d9a7"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.078970 8812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "6341379a5aaf4c0b951fbc91211605a5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "81a0366e99bd17845ca989d00b63e0c9"
server_key_iv: "33125b3a14f081ce02a7da80dde7d9a7"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.080191 8812 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:42.080868 8832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.081029 8812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:42.081115 8812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "6341379a5aaf4c0b951fbc91211605a5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "81a0366e99bd17845ca989d00b63e0c9"
server_key_iv: "33125b3a14f081ce02a7da80dde7d9a7"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.081180 8812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:42.088145 8812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:42.089923 8812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:42.090065 8812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:42.094054 8812 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:43757
I20260808 00:37:42.094090 8922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:43757 every 8 connection(s)
I20260808 00:37:42.094358 8812 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:42.095439 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:42.097856 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: Bootstrap starting.
I20260808 00:37:42.098449 8923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:42.098768 8923 log.cc:824] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:42.099438 8923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: No bootstrap required, opened a new log
I20260808 00:37:42.101063 8923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } }
I20260808 00:37:42.101259 8923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:42.101276 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8812
I20260808 00:37:42.101331 8923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6341379a5aaf4c0b951fbc91211605a5, State: Initialized, Role: FOLLOWER
I20260808 00:37:42.101397 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:42.101491 8923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } }
I20260808 00:37:42.101577 8923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:42.101601 1856 external_mini_cluster.cc:1428] Setting key ab8a1c44b3973dae7683a3fa2149cae3
I20260808 00:37:42.101614 8923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:42.101658 8923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:42.102620 8923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } }
I20260808 00:37:42.102734 8923 leader_election.cc:302] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6341379a5aaf4c0b951fbc91211605a5; no voters:
I20260808 00:37:42.102924 8923 leader_election.cc:288] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:42.102944 8929 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:42.103066 8929 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [term 1 LEADER]: Becoming Leader. State: Replica: 6341379a5aaf4c0b951fbc91211605a5, State: Running, Role: LEADER
I20260808 00:37:42.103189 8923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:42.103197 8929 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } }
I20260808 00:37:42.103693 8930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6341379a5aaf4c0b951fbc91211605a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } } }
I20260808 00:37:42.103821 8930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:42.104315 8940 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:42.104853 8940 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:42.104992 8931 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6341379a5aaf4c0b951fbc91211605a5. Latest consensus state: current_term: 1 leader_uuid: "6341379a5aaf4c0b951fbc91211605a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6341379a5aaf4c0b951fbc91211605a5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43757 } } }
I20260808 00:37:42.105060 8931 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:42.106483 8940 catalog_manager.cc:1347] Generated new cluster ID: ef735b9d25c94b089d826add66b8628b
I20260808 00:37:42.106540 8940 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:42.111517 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.102118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57614 (local address 127.1.208.62:43757)
0808 00:37:42.102292 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.102299 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.102327 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:37:42.102423 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.102428 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.102468 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.102573 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.103341 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.104702 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.105704 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.106288 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.106397 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.108580 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.108584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.108584 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:42.108597 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.110118 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.110443 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.110446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.110446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.110542 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.110717 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.110718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.110718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.110949 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.111101 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.111194 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.111327 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":81,"threads_started":1}
I20260808 00:37:42.117010 8940 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:42.117607 8940 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:42.126698 8940 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: Generated new TSK 0
I20260808 00:37:42.126866 8940 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:42.156104 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:42.238010 8967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:42.238224 8967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:42.238255 8967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:42.239385 8967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:42.239424 8967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:42.239459 8967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:42.240939 8967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:42.241168 8967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:42.241410 8967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:42.246157 8978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.246145 8975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.246145 8980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.246397 8967 server_base.cc:1034] running on GCE node
I20260808 00:37:42.246608 8967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:42.246824 8967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:42.247959 8967 hybrid_clock.cc:648] HybridClock initialized: now 1786149462247947 us; error 29 us; skew 500 ppm
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:42.250305 8967 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:42.250617 8967 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:42.250667 8967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:42.250788 8967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:42.251832 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "7fae24ff29054b6c8bee96591d854aea"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44ca61eb8dd19913aae13572bea859a8"
server_key_iv: "30e2a89901f69e3b3f58841bea562655"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.252167 8967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "7fae24ff29054b6c8bee96591d854aea"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44ca61eb8dd19913aae13572bea859a8"
server_key_iv: "30e2a89901f69e3b3f58841bea562655"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.253391 8967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:42.254040 8990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.254189 8967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:42.254271 8967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "7fae24ff29054b6c8bee96591d854aea"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44ca61eb8dd19913aae13572bea859a8"
server_key_iv: "30e2a89901f69e3b3f58841bea562655"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.254333 8967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:42.290412 8967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:42.292277 8967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:42.292431 8967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:42.292678 8967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:42.292995 8967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:42.293030 8967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.293061 8967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:42.293077 8967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.298678 8967 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35343
I20260808 00:37:42.298734 9177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35343 every 8 connection(s)
I20260808 00:37:42.298980 8967 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:42.301723 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8967
I20260808 00:37:42.301816 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:42.301981 1856 external_mini_cluster.cc:1428] Setting key 6ee04bc1a7fbb33980cb1f5894827382
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:42.308331 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.299739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:53377 (local address 127.1.208.62:43757)
0808 00:37:42.299789 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.299791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.299997 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:37:42.300491 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.300493 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.300507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.300549 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.301579 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.302152 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.303267 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.303580 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.303657 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.305781 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.305783 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.305783 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:42.305791 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.306969 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.307410 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.307411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.307412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.307470 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.307688 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.307689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.307689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.307816 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.307875 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.308148 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.308241 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:42.308499 9185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.299692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43757 (local address 127.1.208.1:53377)
0808 00:37:42.299867 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.299910 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.300395 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.300618 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.300621 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.300900 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:42.301497 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.301504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.302251 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.302252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.303181 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.303184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.303665 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.303666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.303779 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.304247 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:42.304256 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:42.305697 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:42.307115 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.307122 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.307124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.307331 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.307525 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.307526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.307526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.307632 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.307872 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:42.307880 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:42.308025 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.308116 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.308274 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260808 00:37:42.309176 9179 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43757
I20260808 00:37:42.309270 9179 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:42.309445 9179 heartbeater.cc:511] Master 127.1.208.62:43757 requested a full tablet report, sending...
I20260808 00:37:42.309978 8858 ts_manager.cc:194] Registered new tserver with Master: 7fae24ff29054b6c8bee96591d854aea (127.1.208.1:35343)
I20260808 00:37:42.310567 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:53377
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:42.328122 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:42.411545 9188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:42.411744 9188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:42.411767 9188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:42.412900 9188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:42.412941 9188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:42.412968 9188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:42.414518 9188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:42.414752 9188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:42.414969 9188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:42.419866 9197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.419853 9202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.419852 9195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.420174 9188 server_base.cc:1034] running on GCE node
I20260808 00:37:42.420387 9188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:42.420624 9188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:42.421756 9188 hybrid_clock.cc:648] HybridClock initialized: now 1786149462421740 us; error 27 us; skew 500 ppm
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:42.424191 9188 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:42.424525 9188 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:42.424573 9188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:42.424670 9188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:42.425805 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "e6c80bc55a854f5b975ddf0f6d593ff5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44eb3490bb3ab0c2ee2e26d2b156ad37"
server_key_iv: "9ce83639e8f887086a82a96c9bfa790d"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.426244 9188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "e6c80bc55a854f5b975ddf0f6d593ff5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44eb3490bb3ab0c2ee2e26d2b156ad37"
server_key_iv: "9ce83639e8f887086a82a96c9bfa790d"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.427722 9188 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:42.428412 9213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.428568 9188 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:42.428658 9188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "e6c80bc55a854f5b975ddf0f6d593ff5"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "44eb3490bb3ab0c2ee2e26d2b156ad37"
server_key_iv: "9ce83639e8f887086a82a96c9bfa790d"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.428720 9188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:42.442539 9188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:42.444351 9188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:42.444482 9188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:42.444695 9188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:42.445017 9188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:42.445053 9188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.445081 9188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:42.445104 9188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.450846 9188 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:39787
I20260808 00:37:42.450903 9406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:39787 every 8 connection(s)
I20260808 00:37:42.451153 9188 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:42.452942 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 9188
I20260808 00:37:42.453047 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:42.453222 1856 external_mini_cluster.cc:1428] Setting key 6ec11eba91109ae8c4040cf89b7c871d
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:42.460110 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.451803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:53593 (local address 127.1.208.62:43757)
0808 00:37:42.451853 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.451856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.452069 (+ 213us) server_negotiation.cc:407] Connection header received
0808 00:37:42.452588 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.452589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.452600 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.452635 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.453673 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.454234 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.455191 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.455394 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.455449 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.457586 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.457588 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.457588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:42.457595 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.458886 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.459239 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.459241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.459241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.459294 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.459514 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.459515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.459516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.459623 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.459680 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.459940 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.460024 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:42.460275 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.451784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43757 (local address 127.1.208.2:53593)
0808 00:37:42.451950 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.451988 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.452493 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.452698 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.452701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.452959 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:42.453586 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.453594 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.454317 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.454318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.455114 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.455116 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.455463 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.455464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.455589 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.456039 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:42.456047 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:42.457502 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:42.458958 (+ 1456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.458964 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.458966 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.459174 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.459353 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.459354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.459354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.459460 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.459679 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:42.459685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:42.459819 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.459910 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.460061 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260808 00:37:42.460904 9409 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43757
I20260808 00:37:42.461001 9409 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:42.461146 9409 heartbeater.cc:511] Master 127.1.208.62:43757 requested a full tablet report, sending...
I20260808 00:37:42.461539 8858 ts_manager.cc:194] Registered new tserver with Master: e6c80bc55a854f5b975ddf0f6d593ff5 (127.1.208.2:39787)
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
I20260808 00:37:42.461889 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:53593
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:42.479382 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:41793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:42.563129 9420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:42.563340 9420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:42.563364 9420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:42.564538 9420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:42.564581 9420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:42.564610 9420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:42.566208 9420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:42.566465 9420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:42.566689 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:42.571866 9429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.571879 9434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.572153 9425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.572386 9420 server_base.cc:1034] running on GCE node
I20260808 00:37:42.572602 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:42.572853 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:42.573999 9420 hybrid_clock.cc:648] HybridClock initialized: now 1786149462573981 us; error 31 us; skew 500 ppm
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:42.577206 9420 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:42.582082 9420 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:42.582152 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:42.582279 9420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:42.583724 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "44cf7967e5344d9b9955c57fa2621ec9"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "b974ab34c963097d3379a7095e20aace"
server_key_iv: "38778d87e1a4510748594de237df0f45"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.584173 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "44cf7967e5344d9b9955c57fa2621ec9"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "b974ab34c963097d3379a7095e20aace"
server_key_iv: "38778d87e1a4510748594de237df0f45"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.586638 9420 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:42.587639 9446 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.587878 9420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:42.587989 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "44cf7967e5344d9b9955c57fa2621ec9"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "b974ab34c963097d3379a7095e20aace"
server_key_iv: "38778d87e1a4510748594de237df0f45"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.588079 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:42.617687 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:42.620436 9420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:42.620604 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:42.620848 9420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:42.621214 9420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:42.621253 9420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.621284 9420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:42.621330 9420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.627538 9420 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:36221
I20260808 00:37:42.627653 9639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:36221 every 8 connection(s)
I20260808 00:37:42.627912 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:37:42 dist-test-slave-gjw7 krb5kdc[8785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149462, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:42.637463 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 9420
I20260808 00:37:42.637571 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:42.637801 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.628682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:44763 (local address 127.1.208.62:43757)
0808 00:37:42.628741 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.628743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.628963 (+ 220us) server_negotiation.cc:407] Connection header received
0808 00:37:42.629557 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.629558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.629571 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.629615 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.630626 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.631140 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.632248 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.632528 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.632584 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.634821 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.634822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.634822 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:42.634830 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.636176 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.636640 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.636642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.636642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.636715 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.637002 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.637003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.637003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.637110 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.637158 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.637644 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.637732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:42.637811 1856 external_mini_cluster.cc:1428] Setting key 935e811ee349235719538d23740a80e4
I20260808 00:37:42.638038 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.628655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43757 (local address 127.1.208.3:44763)
0808 00:37:42.628831 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.628873 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.629453 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.629691 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.629695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.629985 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:42.630522 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.630530 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.631236 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.631237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.632155 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.632158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.632609 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.632610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.632747 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.633324 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:42.633336 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:42.634729 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:42.636309 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.636317 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.636319 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.636552 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.636803 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:42.636804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:42.636805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:42.636943 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:42.637191 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:42.637242 (+ 51us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:42.637452 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.637603 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.637793 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260808 00:37:42.638556 9640 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43757
I20260808 00:37:42.638661 9640 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:42.638828 9640 heartbeater.cc:511] Master 127.1.208.62:43757 requested a full tablet report, sending...
I20260808 00:37:42.639174 8858 ts_manager.cc:194] Registered new tserver with Master: 44cf7967e5344d9b9955c57fa2621ec9 (127.1.208.3:36221)
I20260808 00:37:42.639565 8858 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:44763
I20260808 00:37:42.649655 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:37:42.656790 8928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.650955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57618 (local address 127.1.208.62:43757)
0808 00:37:42.651003 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.651005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.651101 (+ 96us) server_negotiation.cc:407] Connection header received
0808 00:37:42.651168 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.651169 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.651180 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.651227 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.651780 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.652267 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.653003 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.653193 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.653249 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.654449 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.654450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.654450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:42.654458 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.655602 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.655936 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.655937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.655937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.655997 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:42.656185 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:42.656186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:42.656186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:42.656370 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.656462 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.656580 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.656677 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:42.658286 8858 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57618:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:42.659940 8858 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:42.667037 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.1.208.3:36221)
0808 00:37:42.663274 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.663279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.663519 (+ 240us) server_negotiation.cc:407] Connection header received
0808 00:37:42.663628 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.663632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.663648 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.663716 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:42.663901 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.664602 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.666082 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.666646 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.666711 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.666774 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.666910 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.666949 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":97,"threads_started":1}
I20260808 00:37:42.667034 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36221 (local address 127.0.0.1:59798)
0808 00:37:42.663426 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.663434 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.663476 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.663805 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.663805 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.663807 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:42.663836 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.663837 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.664691 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.664692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.665983 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.665986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.666725 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.666726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.666803 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.666819 (+ 16us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.666885 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.666946 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":714,"mutex_wait_us":26,"thread_start_us":86,"threads_started":1}
I20260808 00:37:42.667042 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:39787 (local address 127.0.0.1:47864)
0808 00:37:42.663197 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.663208 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.663275 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.663604 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.663606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.663608 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:42.663667 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.663669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.665040 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.665041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.665874 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.665877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.666801 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.666802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.666868 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.666873 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.666915 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.666957 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":121,"threads_started":1}
I20260808 00:37:42.667567 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47864 (local address 127.1.208.2:39787)
0808 00:37:42.663180 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.663185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.663267 (+ 82us) server_negotiation.cc:407] Connection header received
0808 00:37:42.663373 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.663376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.663395 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.663495 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:42.663746 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.664925 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.665959 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.666696 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.667082 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.667422 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.667448 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.667486 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":92,"threads_started":1}
I20260808 00:37:42.667837 9526 tablet_service.cc:1467] Processing CreateTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd (DEFAULT_TABLE table=test-table [id=142bfe6d6e32459fb44597bf7a3e031d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:42.668207 9526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edb22a7fd0b4778880c5b6599a14bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:42.668233 9298 tablet_service.cc:1467] Processing CreateTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd (DEFAULT_TABLE table=test-table [id=142bfe6d6e32459fb44597bf7a3e031d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:42.668643 9298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edb22a7fd0b4778880c5b6599a14bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:42.670944 9675 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Bootstrap starting.
I20260808 00:37:42.671224 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.1.208.1:35343)
0808 00:37:42.662941 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.662946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.663087 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:37:42.663445 (+ 358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.663448 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.663463 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.663542 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:42.663751 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.664891 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.666249 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.666963 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.667022 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.667075 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.671085 (+ 4010us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.671142 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260808 00:37:42.671480 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.662634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35343 (local address 127.0.0.1:55062)
0808 00:37:42.662978 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.663022 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.663356 (+ 334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.663570 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.663573 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.663576 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:42.663655 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.663658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.664996 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.664997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.666174 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.666177 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.667226 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.667227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.667311 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.667316 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.671351 (+ 4035us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.671407 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":79,"threads_started":1}
I20260808 00:37:42.671914 9675 tablet_bootstrap.cc:654] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:42.672259 9071 tablet_service.cc:1467] Processing CreateTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd (DEFAULT_TABLE table=test-table [id=142bfe6d6e32459fb44597bf7a3e031d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:42.672539 9675 log.cc:824] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:42.672629 9071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4edb22a7fd0b4778880c5b6599a14bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:42.672932 9676 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Bootstrap starting.
I20260808 00:37:42.673386 9675 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: No bootstrap required, opened a new log
I20260808 00:37:42.673454 9675 ts_tablet_manager.cc:1397] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:37:42.673617 9676 tablet_bootstrap.cc:654] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:42.674196 9676 log.cc:824] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:42.675020 9676 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: No bootstrap required, opened a new log
I20260808 00:37:42.675097 9676 ts_tablet_manager.cc:1397] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:37:42.675227 9675 raft_consensus.cc:348] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.675412 9675 raft_consensus.cc:374] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:42.675454 9675 raft_consensus.cc:729] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44cf7967e5344d9b9955c57fa2621ec9, State: Initialized, Role: FOLLOWER
I20260808 00:37:42.675570 9675 consensus_queue.cc:260] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.675858 9675 ts_tablet_manager.cc:1428] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:42.675875 9640 heartbeater.cc:503] Master 127.1.208.62:43757 was elected leader, sending a full tablet report...
I20260808 00:37:42.676327 9682 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: Bootstrap starting.
I20260808 00:37:42.677053 9682 tablet_bootstrap.cc:654] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:42.677258 9676 raft_consensus.cc:348] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.677485 9676 raft_consensus.cc:374] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:42.677515 9676 raft_consensus.cc:729] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c80bc55a854f5b975ddf0f6d593ff5, State: Initialized, Role: FOLLOWER
I20260808 00:37:42.677631 9676 consensus_queue.cc:260] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.677701 9682 log.cc:824] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:42.677932 9409 heartbeater.cc:503] Master 127.1.208.62:43757 was elected leader, sending a full tablet report...
I20260808 00:37:42.677913 9676 ts_tablet_manager.cc:1428] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:37:42.678483 9682 tablet_bootstrap.cc:492] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: No bootstrap required, opened a new log
I20260808 00:37:42.678556 9682 ts_tablet_manager.cc:1397] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:42.680096 9682 raft_consensus.cc:348] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.680231 9682 raft_consensus.cc:374] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:42.680265 9682 raft_consensus.cc:729] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fae24ff29054b6c8bee96591d854aea, State: Initialized, Role: FOLLOWER
I20260808 00:37:42.680377 9682 consensus_queue.cc:260] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.680635 9682 ts_tablet_manager.cc:1428] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:37:42.680673 9179 heartbeater.cc:503] Master 127.1.208.62:43757 was elected leader, sending a full tablet report...
W20260808 00:37:42.701669 9410 tablet.cc:2369] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:42.756107 9681 raft_consensus.cc:482] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:42.756242 9681 raft_consensus.cc:504] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.756613 9681 leader_election.cc:288] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7fae24ff29054b6c8bee96591d854aea (127.1.208.1:35343), e6c80bc55a854f5b975ddf0f6d593ff5 (127.1.208.2:39787)
I20260808 00:37:42.760654 9644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.756793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35343 (local address 127.1.208.3:47963)
0808 00:37:42.756858 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.756870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.756940 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.757143 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.757145 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.757147 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:42.757224 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.757226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.758222 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.758223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.759497 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.759501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.760312 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.760313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.760435 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.760441 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.760502 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.760555 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":43}
I20260808 00:37:42.760670 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.756939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:47963 (local address 127.1.208.1:35343)
0808 00:37:42.756989 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.756991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.757001 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:42.757032 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.757033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.757046 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.757114 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:42.757323 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.758138 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.759609 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.760202 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.760293 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.760307 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.760541 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.760593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:37:42.760670 9670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.757015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:41297 (local address 127.1.208.2:39787)
0808 00:37:42.757066 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.757068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.757156 (+ 88us) server_negotiation.cc:407] Connection header received
0808 00:37:42.757199 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.757200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.757215 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.757255 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:42.757414 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.758131 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.759614 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.760226 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.760353 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.760365 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.760541 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.760593 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:42.760661 9689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.756846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:39787 (local address 127.1.208.3:41297)
0808 00:37:42.757053 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.757062 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:42.757147 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:42.757274 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:42.757275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:42.757276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:42.757347 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.757349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.758212 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.758214 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.759497 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:42.759501 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.760356 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.760357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:42.760435 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.760441 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:37:42.760502 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:37:42.760555 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":29,"thread_start_us":130,"threads_started":1}
I20260808 00:37:42.761132 9334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edb22a7fd0b4778880c5b6599a14bcd" candidate_uuid: "44cf7967e5344d9b9955c57fa2621ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" is_pre_election: true
I20260808 00:37:42.761140 9107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edb22a7fd0b4778880c5b6599a14bcd" candidate_uuid: "44cf7967e5344d9b9955c57fa2621ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fae24ff29054b6c8bee96591d854aea" is_pre_election: true
I20260808 00:37:42.761320 9334 raft_consensus.cc:2454] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44cf7967e5344d9b9955c57fa2621ec9 in term 0.
I20260808 00:37:42.761322 9107 raft_consensus.cc:2454] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44cf7967e5344d9b9955c57fa2621ec9 in term 0.
I20260808 00:37:42.761583 9448 leader_election.cc:302] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44cf7967e5344d9b9955c57fa2621ec9, 7fae24ff29054b6c8bee96591d854aea; no voters:
I20260808 00:37:42.761739 9681 raft_consensus.cc:2781] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:42.761791 9681 raft_consensus.cc:482] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:42.761808 9681 raft_consensus.cc:3037] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:42.762727 9681 raft_consensus.cc:504] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.762876 9681 leader_election.cc:288] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [CANDIDATE]: Term 1 election: Requested vote from peers 7fae24ff29054b6c8bee96591d854aea (127.1.208.1:35343), e6c80bc55a854f5b975ddf0f6d593ff5 (127.1.208.2:39787)
I20260808 00:37:42.763079 9107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edb22a7fd0b4778880c5b6599a14bcd" candidate_uuid: "44cf7967e5344d9b9955c57fa2621ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7fae24ff29054b6c8bee96591d854aea"
I20260808 00:37:42.763173 9107 raft_consensus.cc:3037] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:42.763130 9334 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4edb22a7fd0b4778880c5b6599a14bcd" candidate_uuid: "44cf7967e5344d9b9955c57fa2621ec9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5"
I20260808 00:37:42.763310 9334 raft_consensus.cc:3037] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:42.763953 9107 raft_consensus.cc:2454] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44cf7967e5344d9b9955c57fa2621ec9 in term 1.
I20260808 00:37:42.764127 9448 leader_election.cc:302] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44cf7967e5344d9b9955c57fa2621ec9, 7fae24ff29054b6c8bee96591d854aea; no voters:
I20260808 00:37:42.764173 9334 raft_consensus.cc:2454] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44cf7967e5344d9b9955c57fa2621ec9 in term 1.
I20260808 00:37:42.764225 9681 raft_consensus.cc:2781] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:42.764394 9681 raft_consensus.cc:686] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 1 LEADER]: Becoming Leader. State: Replica: 44cf7967e5344d9b9955c57fa2621ec9, State: Running, Role: LEADER
I20260808 00:37:42.764493 9681 consensus_queue.cc:237] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } }
I20260808 00:37:42.765471 8858 catalog_manager.cc:5495] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44cf7967e5344d9b9955c57fa2621ec9 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "44cf7967e5344d9b9955c57fa2621ec9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44cf7967e5344d9b9955c57fa2621ec9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36221 } health_report { overall_health: HEALTHY } } }
I20260808 00:37:42.783099 9674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.780644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.1.208.3:36221)
0808 00:37:42.780712 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.780714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.780723 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:42.780772 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.780772 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.780794 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.780859 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:42.780999 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.781555 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.782329 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:42.782522 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:42.782582 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:42.782749 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:42.782872 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:42.782950 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.782967 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.782993 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:37:42.785151 9107 raft_consensus.cc:1261] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 1 FOLLOWER]: Refusing update from remote peer 44cf7967e5344d9b9955c57fa2621ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:42.785190 9334 raft_consensus.cc:1261] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 1 FOLLOWER]: Refusing update from remote peer 44cf7967e5344d9b9955c57fa2621ec9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:42.785640 9690 consensus_queue.cc:1035] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6c80bc55a854f5b975ddf0f6d593ff5" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 39787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:42.785797 9681 consensus_queue.cc:1035] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fae24ff29054b6c8bee96591d854aea" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:42.788688 9696 mvcc.cc:205] Tried to move back new op lower bound from 7316068199565148160 to 7316068199484055552. Current Snapshot: MvccSnapshot[applied={T|T < 7316068199565148160}]
I20260808 00:37:42.788995 9695 mvcc.cc:205] Tried to move back new op lower bound from 7316068199565148160 to 7316068199484055552. Current Snapshot: MvccSnapshot[applied={T|T < 7316068199565148160}]
I20260808 00:37:42.791359 8858 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:42.791451 8858 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57618:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:42.792196 8858 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6341379a5aaf4c0b951fbc91211605a5: Sending DeleteTablet for 3 replicas of tablet 4edb22a7fd0b4778880c5b6599a14bcd
I20260808 00:37:42.792578 9071 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:55062
I20260808 00:37:42.792578 9298 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:47864
I20260808 00:37:42.792869 9708 tablet_replica.cc:330] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: stopping tablet replica
I20260808 00:37:42.792977 9708 raft_consensus.cc:2229] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:42.792946 9526 tablet_service.cc:1514] Processing DeleteTablet for tablet 4edb22a7fd0b4778880c5b6599a14bcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59798
I20260808 00:37:42.793078 9708 raft_consensus.cc:2258] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:42.793272 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8967
I20260808 00:37:42.793340 9706 tablet_replica.cc:330] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea: stopping tablet replica
I20260808 00:37:42.793424 9706 raft_consensus.cc:2229] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:42.793519 9706 raft_consensus.cc:2258] T 4edb22a7fd0b4778880c5b6599a14bcd P 7fae24ff29054b6c8bee96591d854aea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:42.805142 9708 ts_tablet_manager.cc:1905] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:42.807052 9708 ts_tablet_manager.cc:1918] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:42.807157 9708 log.cc:1196] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/4edb22a7fd0b4778880c5b6599a14bcd
I20260808 00:37:42.807526 9708 ts_tablet_manager.cc:1939] T 4edb22a7fd0b4778880c5b6599a14bcd P e6c80bc55a854f5b975ddf0f6d593ff5: Deleting consensus metadata
I20260808 00:37:42.808130 8833 catalog_manager.cc:4841] TS e6c80bc55a854f5b975ddf0f6d593ff5 (127.1.208.2:39787): tablet 4edb22a7fd0b4778880c5b6599a14bcd (table test-table [id=142bfe6d6e32459fb44597bf7a3e031d]) successfully deleted
I20260808 00:37:42.808542 9709 tablet_replica.cc:330] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: stopping tablet replica
I20260808 00:37:42.808679 9709 raft_consensus.cc:2229] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:42.808841 9709 raft_consensus.cc:2258] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:42.809353 9709 ts_tablet_manager.cc:1905] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:42.810964 9709 ts_tablet_manager.cc:1918] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:42.811046 9709 log.cc:1196] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/4edb22a7fd0b4778880c5b6599a14bcd
I20260808 00:37:42.811337 9709 ts_tablet_manager.cc:1939] T 4edb22a7fd0b4778880c5b6599a14bcd P 44cf7967e5344d9b9955c57fa2621ec9: Deleting consensus metadata
I20260808 00:37:42.811789 8836 catalog_manager.cc:4841] TS 44cf7967e5344d9b9955c57fa2621ec9 (127.1.208.3:36221): tablet 4edb22a7fd0b4778880c5b6599a14bcd (table test-table [id=142bfe6d6e32459fb44597bf7a3e031d]) successfully deleted
W20260808 00:37:42.813341 8834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:35343 (error 108)
I20260808 00:37:42.813905 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.813761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35343 (local address 127.0.0.1:55066)
0808 00:37:42.813820 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:42.813864 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:37:42.813961 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 9188
W20260808 00:37:42.826560 8834 catalog_manager.cc:4584] TS 7fae24ff29054b6c8bee96591d854aea (127.1.208.1:35343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4edb22a7fd0b4778880c5b6599a14bcd: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35343: connect: Connection refused (error 111)
I20260808 00:37:42.835604 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 9420
I20260808 00:37:42.849828 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8812
2026-08-08T00:37:42Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2177 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T00:37:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:42Z Disabled control of system clock
I20260808 00:37:42.887814 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:41033
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:41033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:37:42.968835 9717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:42.969044 9717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:42.969064 9717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:42.970211 9717 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:42.970256 9717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:42.970268 9717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:42.970290 9717 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:42.970312 9717 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:42.971704 9717 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:41033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9717
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:42.971948 9717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:42.972237 9717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:42.976724 9726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:42.976728 9729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.976758 9717 server_base.cc:1034] running on GCE node
W20260808 00:37:42.976759 9724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:42.977141 9717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:42.977433 9717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:42.978575 9717 hybrid_clock.cc:648] HybridClock initialized: now 1786149462978559 us; error 29 us; skew 500 ppm
I20260808 00:37:42.979750 9717 webserver.cc:460] Webserver started at http://127.1.208.62:46503/ using document root <none> and password file <none>
I20260808 00:37:42.979977 9717 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:42.980021 9717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:42.980144 9717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:42.981161 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "4178ca105f8d44e79a1b67c7cdd3e571"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "fba3fd3ec1ff880da9bd0d6e755a4694"
server_key_iv: "35162199cc8b37a0fb23e1e32bee06b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.981537 9717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "4178ca105f8d44e79a1b67c7cdd3e571"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "fba3fd3ec1ff880da9bd0d6e755a4694"
server_key_iv: "35162199cc8b37a0fb23e1e32bee06b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.982739 9717 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:42.983385 9740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:42.983528 9717 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:42.983610 9717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "4178ca105f8d44e79a1b67c7cdd3e571"
format_stamp: "Formatted at 2026-08-08 00:37:42 on dist-test-slave-gjw7"
server_key: "fba3fd3ec1ff880da9bd0d6e755a4694"
server_key_iv: "35162199cc8b37a0fb23e1e32bee06b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:42.983670 9717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:42.986862 9717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:42.987097 9717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:42.987208 9717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:42.990893 9717 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:41033
I20260808 00:37:42.990947 9839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:41033 every 8 connection(s)
I20260808 00:37:42.991214 9717 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:42.991797 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 9717
I20260808 00:37:42.991889 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:42.992053 1856 external_mini_cluster.cc:1428] Setting key d189d714ebd5a227839727445f706cbe
I20260808 00:37:42.992272 9841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:42.994540 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:42.992503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.1.208.62:41033)
0808 00:37:42.992679 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:42.992685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:42.992708 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:37:42.993365 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:42.993373 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:42.993414 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:42.993520 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:42.994137 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:42.994140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:42.994140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:42.994144 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:42.994179 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:42.994189 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:42.994239 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:42.994315 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:37:42.994367 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260808 00:37:42.995436 9841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: Bootstrap starting.
I20260808 00:37:42.995991 9841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:42.996232 9841 log.cc:824] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:42.996819 9841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: No bootstrap required, opened a new log
I20260808 00:37:42.998193 9841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } }
I20260808 00:37:42.998322 9841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:42.998343 9841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4178ca105f8d44e79a1b67c7cdd3e571, State: Initialized, Role: FOLLOWER
I20260808 00:37:42.998463 9841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } }
I20260808 00:37:42.998539 9841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:42.998570 9841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:42.998616 9841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:42.999082 9841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } }
I20260808 00:37:42.999171 9841 leader_election.cc:302] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4178ca105f8d44e79a1b67c7cdd3e571; no voters:
I20260808 00:37:42.999325 9841 leader_election.cc:288] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:42.999501 9849 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:42.999593 9841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:42.999616 9849 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [term 1 LEADER]: Becoming Leader. State: Replica: 4178ca105f8d44e79a1b67c7cdd3e571, State: Running, Role: LEADER
I20260808 00:37:42.999709 9849 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } }
I20260808 00:37:43.000053 9852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4178ca105f8d44e79a1b67c7cdd3e571. Latest consensus state: current_term: 1 leader_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } } }
I20260808 00:37:43.000151 9852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:43.000313 9850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4178ca105f8d44e79a1b67c7cdd3e571" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 41033 } } }
I20260808 00:37:43.000370 9850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:43.000931 9861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:43.001382 9861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:43.002745 9861 catalog_manager.cc:1347] Generated new cluster ID: 0e772ce283484680995976097b8bfa48
I20260808 00:37:43.002794 9861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:43.029464 9861 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:43.029651 9861 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:43.032502 9861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: Generated new TSK 0
I20260808 00:37:43.032675 9861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:43.038189 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--builtin_ntp_servers=127.1.208.20:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:37:43.122344 9877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:43.122574 9877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:43.122605 9877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:43.123770 9877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:43.123813 9877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:43.123852 9877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:43.125207 9877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:43.125492 9877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:43.125788 9877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:43.130702 9890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.130756 9887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.130900 9877 server_base.cc:1034] running on GCE node
W20260808 00:37:43.130947 9893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.131183 9877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:43.131469 9877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:43.132612 9877 hybrid_clock.cc:648] HybridClock initialized: now 1786149463132596 us; error 28 us; skew 500 ppm
I20260808 00:37:43.133961 9877 webserver.cc:460] Webserver started at http://127.1.208.1:44331/ using document root <none> and password file <none>
I20260808 00:37:43.134182 9877 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:43.134274 9877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:43.134402 9877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:43.135453 9877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "5a7d1874f54f48e4acf83625779fb422"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "c52db956c1f0239613429e8d18c2296b"
server_key_iv: "4480af449d4a2225c139ba3e446ea8db"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.135790 9877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "5a7d1874f54f48e4acf83625779fb422"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "c52db956c1f0239613429e8d18c2296b"
server_key_iv: "4480af449d4a2225c139ba3e446ea8db"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.136993 9877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:37:43.137756 9905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.137909 9877 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.138006 9877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "5a7d1874f54f48e4acf83625779fb422"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "c52db956c1f0239613429e8d18c2296b"
server_key_iv: "4480af449d4a2225c139ba3e446ea8db"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.138065 9877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:43.141217 9877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:43.141481 9877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:43.141603 9877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:43.141784 9877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:43.142062 9877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:43.142102 9877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.142127 9877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:43.142151 9877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.147708 9877 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:36173
I20260808 00:37:43.147769 10124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:36173 every 8 connection(s)
I20260808 00:37:43.148043 9877 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:43.151048 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.148685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:35933 (local address 127.1.208.62:41033)
0808 00:37:43.148736 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.148737 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.148960 (+ 223us) server_negotiation.cc:407] Connection header received
0808 00:37:43.149532 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.149534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.149541 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.149600 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.150822 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.150824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.150824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.150828 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.150844 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.150848 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.150897 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.150964 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.150978 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:37:43.151232 10131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.148663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41033 (local address 127.1.208.1:35933)
0808 00:37:43.148850 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.148881 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.149434 (+ 553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.149669 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.149673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.149683 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.150663 (+ 980us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.150663 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.150706 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.150734 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.150899 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.150905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.150909 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.150977 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.151037 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 00:37:43.151707 10126 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41033
I20260808 00:37:43.151787 10126 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:43.151926 10126 heartbeater.cc:511] Master 127.1.208.62:41033 requested a full tablet report, sending...
I20260808 00:37:43.152396 9773 ts_manager.cc:194] Registered new tserver with Master: 5a7d1874f54f48e4acf83625779fb422 (127.1.208.1:36173)
I20260808 00:37:43.154074 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 9877
I20260808 00:37:43.154172 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:43.154345 1856 external_mini_cluster.cc:1428] Setting key ef07937cebda09bc3968b4a732e80341
I20260808 00:37:43.155126 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--builtin_ntp_servers=127.1.208.20:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:37:43.256102 10132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:43.256448 10132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:43.256524 10132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:43.257970 10132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:43.258020 10132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:43.258059 10132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:43.259567 10132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:43.260540 10132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:43.261247 10132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:43.268877 10144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.268887 10145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.268918 10132 server_base.cc:1034] running on GCE node
W20260808 00:37:43.268887 10147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.269387 10132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:43.269629 10132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:43.270793 10132 hybrid_clock.cc:648] HybridClock initialized: now 1786149463270785 us; error 34 us; skew 500 ppm
I20260808 00:37:43.272504 10132 webserver.cc:460] Webserver started at http://127.1.208.2:44803/ using document root <none> and password file <none>
I20260808 00:37:43.272792 10132 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:43.272845 10132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:43.273075 10132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:43.274595 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "a0e1fb3ee4404de9871b2f251bced24e"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "653b7953c7f1698b1b2e791d1b16ae41"
server_key_iv: "246b9cff3e033a051be6ec303441a9ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.274972 10132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "a0e1fb3ee4404de9871b2f251bced24e"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "653b7953c7f1698b1b2e791d1b16ae41"
server_key_iv: "246b9cff3e033a051be6ec303441a9ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.277146 10132 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 00:37:43.278403 10164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.278855 10132 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:43.278947 10132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "a0e1fb3ee4404de9871b2f251bced24e"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "653b7953c7f1698b1b2e791d1b16ae41"
server_key_iv: "246b9cff3e033a051be6ec303441a9ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.279006 10132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:43.285135 10132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:43.285552 10132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:43.285730 10132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:43.285920 10132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:43.286644 10132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:43.286720 10132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.286753 10132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:43.286775 10132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.293942 10132 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:38255
I20260808 00:37:43.294015 10367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:38255 every 8 connection(s)
I20260808 00:37:43.294293 10132 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:43.298465 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.295140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:35309 (local address 127.1.208.62:41033)
0808 00:37:43.295234 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.295244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.295534 (+ 290us) server_negotiation.cc:407] Connection header received
0808 00:37:43.296262 (+ 728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.296263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.296273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.296342 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.298092 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.298094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.298094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.298107 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.298178 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.298184 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.298237 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.298305 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.298318 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:37:43.298802 10371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.295139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41033 (local address 127.1.208.2:35309)
0808 00:37:43.295380 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.295431 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.296155 (+ 724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.296475 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.296479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.296488 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.297907 (+ 1419us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.297907 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.297939 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.297999 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.298245 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.298249 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.298251 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.298307 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.298453 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":113,"threads_started":1}
I20260808 00:37:43.299583 10368 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41033
I20260808 00:37:43.299685 10368 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:43.299885 10368 heartbeater.cc:511] Master 127.1.208.62:41033 requested a full tablet report, sending...
I20260808 00:37:43.300690 9773 ts_manager.cc:194] Registered new tserver with Master: a0e1fb3ee4404de9871b2f251bced24e (127.1.208.2:38255)
I20260808 00:37:43.301416 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 10132
I20260808 00:37:43.301528 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:43.301712 1856 external_mini_cluster.cc:1428] Setting key 4f115379eddb43a131045337313c846b
I20260808 00:37:43.303025 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--builtin_ntp_servers=127.1.208.20:41317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 00:37:43.404488 10372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:43.404687 10372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:43.404707 10372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:43.405844 10372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:43.405879 10372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:43.405905 10372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:43.407200 10372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:41033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:43.407389 10372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:43.407631 10372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:43.412475 10381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.412506 10380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.412487 10384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.412747 10372 server_base.cc:1034] running on GCE node
I20260808 00:37:43.412945 10372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:43.413168 10372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:43.414338 10372 hybrid_clock.cc:648] HybridClock initialized: now 1786149463414315 us; error 34 us; skew 500 ppm
I20260808 00:37:43.415653 10372 webserver.cc:460] Webserver started at http://127.1.208.3:38883/ using document root <none> and password file <none>
I20260808 00:37:43.415890 10372 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:43.415962 10372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:43.416095 10372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:43.417126 10372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "4edbdab4425a4865ac47e72440245d75"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "aec7c9c65208183a249f1d0cf0e93307"
server_key_iv: "1142b5da1c2a3e88b353f1ef13ae1772"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.417501 10372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "4edbdab4425a4865ac47e72440245d75"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "aec7c9c65208183a249f1d0cf0e93307"
server_key_iv: "1142b5da1c2a3e88b353f1ef13ae1772"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.418780 10372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:43.419492 10398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.419656 10372 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:43.419739 10372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "4edbdab4425a4865ac47e72440245d75"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "aec7c9c65208183a249f1d0cf0e93307"
server_key_iv: "1142b5da1c2a3e88b353f1ef13ae1772"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.419802 10372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:43.423123 10372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:43.423393 10372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:43.423516 10372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:43.423705 10372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:43.423995 10372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:43.424028 10372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.424053 10372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:43.424078 10372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.430308 10372 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:45457
I20260808 00:37:43.430377 10587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:45457 every 8 connection(s)
I20260808 00:37:43.430670 10372 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:43.433549 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.431349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:36461 (local address 127.1.208.62:41033)
0808 00:37:43.431407 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.431409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.431633 (+ 224us) server_negotiation.cc:407] Connection header received
0808 00:37:43.432113 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.432114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.432122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.432174 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.433275 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.433277 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.433278 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.433282 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.433323 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.433327 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.433378 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.433460 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.433475 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:37:43.433717 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.431353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:41033 (local address 127.1.208.3:36461)
0808 00:37:43.431518 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.431551 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.432011 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.432260 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.432264 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.432272 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.433126 (+ 854us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.433127 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.433155 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.433182 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.433387 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.433393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.433396 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.433465 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.433528 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260808 00:37:43.434170 10588 heartbeater.cc:347] Connected to a master server at 127.1.208.62:41033
I20260808 00:37:43.434255 10588 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:43.434379 10588 heartbeater.cc:511] Master 127.1.208.62:41033 requested a full tablet report, sending...
I20260808 00:37:43.434706 9773 ts_manager.cc:194] Registered new tserver with Master: 4edbdab4425a4865ac47e72440245d75 (127.1.208.3:45457)
I20260808 00:37:43.440282 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 10372
I20260808 00:37:43.440375 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:43.440531 1856 external_mini_cluster.cc:1428] Setting key 84ede3ec782232100eb53726dac3192d
I20260808 00:37:43.441490 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:37:43.443930 9845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.442842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39944 (local address 127.1.208.62:41033)
0808 00:37:43.442899 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.442901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.443032 (+ 131us) server_negotiation.cc:407] Connection header received
0808 00:37:43.443105 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.443106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.443113 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.443172 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.443701 (+ 529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.443703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.443703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.443707 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.443721 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.443726 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.443780 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.443844 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.443859 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:43.445416 9773 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:43.447290 9773 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:43.451279 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38255 (local address 127.0.0.1:48546)
0808 00:37:43.449601 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.449646 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.449722 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.450232 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.450240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.450247 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.450723 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.450724 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.450742 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.450767 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.451134 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.451135 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.451136 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.451185 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.451197 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":108,"threads_started":1}
I20260808 00:37:43.451299 10619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48546 (local address 127.1.208.2:38255)
0808 00:37:43.450061 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.450064 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.450075 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:43.450105 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.450107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.450117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.450467 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.451007 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.451011 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.451011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.451015 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.451046 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.451071 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.451140 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.451191 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.451207 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":138,"threads_started":1}
I20260808 00:37:43.451373 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.1.208.1:36173)
0808 00:37:43.450128 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.450133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.450142 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:43.450185 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.450187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.450197 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.450480 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.450946 (+ 466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.450952 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.450953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.450957 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.450993 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.451017 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.451076 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.451250 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.451267 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":121,"threads_started":1}
I20260808 00:37:43.451411 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53202 (local address 127.1.208.3:45457)
0808 00:37:43.449930 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.449934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.450185 (+ 251us) server_negotiation.cc:407] Connection header received
0808 00:37:43.450211 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.450213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.450223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.450286 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.451099 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.451107 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.451107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.451111 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.451142 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.451166 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.451213 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.451313 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.451327 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":88,"threads_started":1}
I20260808 00:37:43.451537 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:45457 (local address 127.0.0.1:53202)
0808 00:37:43.450081 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.450090 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.450132 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.450593 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.450594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.450596 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.451012 (+ 416us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.451012 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.451017 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.451025 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.451234 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.451235 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.451235 (+ 0us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.451272 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.451282 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":83,"threads_started":1}
I20260808 00:37:43.451871 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.449453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:36173 (local address 127.0.0.1:51010)
0808 00:37:43.449944 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.449954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.450001 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.450299 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.450300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.450303 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.450853 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.450853 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.450859 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.450869 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.451075 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.451083 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.451086 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.451158 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.451183 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":4,"thread_start_us":60,"threads_started":1}
I20260808 00:37:43.452425 10244 tablet_service.cc:1467] Processing CreateTablet for tablet 5217a4a85344464b93a2344969c4252a (DEFAULT_TABLE table=test-table [id=a93f80a4b4c24071a061b1ccb0858b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:43.452421 10477 tablet_service.cc:1467] Processing CreateTablet for tablet 5217a4a85344464b93a2344969c4252a (DEFAULT_TABLE table=test-table [id=a93f80a4b4c24071a061b1ccb0858b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:43.452560 10007 tablet_service.cc:1467] Processing CreateTablet for tablet 5217a4a85344464b93a2344969c4252a (DEFAULT_TABLE table=test-table [id=a93f80a4b4c24071a061b1ccb0858b12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:43.452836 10477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5217a4a85344464b93a2344969c4252a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:43.452836 10244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5217a4a85344464b93a2344969c4252a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:43.452870 10007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5217a4a85344464b93a2344969c4252a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:43.455622 10623 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Bootstrap starting.
I20260808 00:37:43.456259 10627 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: Bootstrap starting.
I20260808 00:37:43.456434 10623 tablet_bootstrap.cc:654] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:43.456784 10623 log.cc:824] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:43.457008 10627 tablet_bootstrap.cc:654] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:43.457355 10627 log.cc:824] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:43.457448 10629 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Bootstrap starting.
I20260808 00:37:43.457867 10623 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: No bootstrap required, opened a new log
I20260808 00:37:43.457958 10623 ts_tablet_manager.cc:1397] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:43.458150 10629 tablet_bootstrap.cc:654] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:43.458302 10627 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: No bootstrap required, opened a new log
I20260808 00:37:43.458379 10627 ts_tablet_manager.cc:1397] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:43.458472 10629 log.cc:824] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:43.459087 10629 tablet_bootstrap.cc:492] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: No bootstrap required, opened a new log
I20260808 00:37:43.459144 10629 ts_tablet_manager.cc:1397] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:43.460822 10623 raft_consensus.cc:348] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.461033 10623 raft_consensus.cc:374] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:43.460963 10629 raft_consensus.cc:348] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.460973 10627 raft_consensus.cc:348] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.461093 10623 raft_consensus.cc:729] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4edbdab4425a4865ac47e72440245d75, State: Initialized, Role: FOLLOWER
I20260808 00:37:43.461120 10627 raft_consensus.cc:374] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:43.461117 10629 raft_consensus.cc:374] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:43.461162 10629 raft_consensus.cc:729] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0e1fb3ee4404de9871b2f251bced24e, State: Initialized, Role: FOLLOWER
I20260808 00:37:43.461163 10627 raft_consensus.cc:729] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a7d1874f54f48e4acf83625779fb422, State: Initialized, Role: FOLLOWER
I20260808 00:37:43.461252 10623 consensus_queue.cc:260] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.461283 10627 consensus_queue.cc:260] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.461318 10629 consensus_queue.cc:260] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.461573 10623 ts_tablet_manager.cc:1428] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260808 00:37:43.461573 10627 ts_tablet_manager.cc:1428] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 00:37:43.461597 10629 ts_tablet_manager.cc:1428] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:37:43.461715 10368 heartbeater.cc:503] Master 127.1.208.62:41033 was elected leader, sending a full tablet report...
I20260808 00:37:43.461724 10588 heartbeater.cc:503] Master 127.1.208.62:41033 was elected leader, sending a full tablet report...
I20260808 00:37:43.461728 10126 heartbeater.cc:503] Master 127.1.208.62:41033 was elected leader, sending a full tablet report...
I20260808 00:37:43.474538 10646 raft_consensus.cc:482] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:43.474642 10646 raft_consensus.cc:504] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.474982 10646 leader_election.cc:288] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5a7d1874f54f48e4acf83625779fb422 (127.1.208.1:36173), a0e1fb3ee4404de9871b2f251bced24e (127.1.208.2:38255)
I20260808 00:37:43.476142 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.475218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:37313 (local address 127.1.208.1:36173)
0808 00:37:43.475297 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.475298 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.475306 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:43.475329 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.475330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.475334 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.475380 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.475910 (+ 530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.475911 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.475912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.475916 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.475946 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.475951 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.476008 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.476072 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.476087 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:43.476143 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.475145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:36173 (local address 127.1.208.3:37313)
0808 00:37:43.475201 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.475212 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.475250 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.475392 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.475393 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.475395 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.475812 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.475813 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.475821 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.475835 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.476003 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.476003 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.476004 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.476045 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.476079 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:37:43.476326 10619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.475203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:39373 (local address 127.1.208.2:38255)
0808 00:37:43.475249 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.475251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.475398 (+ 147us) server_negotiation.cc:407] Connection header received
0808 00:37:43.475441 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.475442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.475447 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.475495 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.476076 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.476078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.476078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.476084 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.476102 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.476107 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.476160 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.476255 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.476269 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:43.476512 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.475145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38255 (local address 127.1.208.3:39373)
0808 00:37:43.475318 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.475343 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.475392 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.475505 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.475506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.475508 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:43.475966 (+ 458us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:37:43.475966 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:43.475973 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:37:43.475984 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:43.476169 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:43.476170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:43.476171 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:43.476409 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 00:37:43.476424 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":13,"thread_start_us":118,"threads_started":1}
I20260808 00:37:43.476606 10044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5217a4a85344464b93a2344969c4252a" candidate_uuid: "4edbdab4425a4865ac47e72440245d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7d1874f54f48e4acf83625779fb422" is_pre_election: true
I20260808 00:37:43.476764 10044 raft_consensus.cc:2454] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4edbdab4425a4865ac47e72440245d75 in term 0.
I20260808 00:37:43.476816 10280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5217a4a85344464b93a2344969c4252a" candidate_uuid: "4edbdab4425a4865ac47e72440245d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e1fb3ee4404de9871b2f251bced24e" is_pre_election: true
I20260808 00:37:43.476948 10280 raft_consensus.cc:2454] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4edbdab4425a4865ac47e72440245d75 in term 0.
I20260808 00:37:43.477044 10402 leader_election.cc:302] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4edbdab4425a4865ac47e72440245d75, 5a7d1874f54f48e4acf83625779fb422; no voters:
I20260808 00:37:43.477203 10646 raft_consensus.cc:2781] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:43.477258 10646 raft_consensus.cc:482] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:43.477314 10646 raft_consensus.cc:3037] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:43.478036 10646 raft_consensus.cc:504] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.478166 10646 leader_election.cc:288] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [CANDIDATE]: Term 1 election: Requested vote from peers 5a7d1874f54f48e4acf83625779fb422 (127.1.208.1:36173), a0e1fb3ee4404de9871b2f251bced24e (127.1.208.2:38255)
I20260808 00:37:43.478283 10044 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5217a4a85344464b93a2344969c4252a" candidate_uuid: "4edbdab4425a4865ac47e72440245d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5a7d1874f54f48e4acf83625779fb422"
I20260808 00:37:43.478325 10280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5217a4a85344464b93a2344969c4252a" candidate_uuid: "4edbdab4425a4865ac47e72440245d75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0e1fb3ee4404de9871b2f251bced24e"
I20260808 00:37:43.478364 10044 raft_consensus.cc:3037] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:43.478387 10280 raft_consensus.cc:3037] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:43.479243 10044 raft_consensus.cc:2454] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4edbdab4425a4865ac47e72440245d75 in term 1.
I20260808 00:37:43.479243 10280 raft_consensus.cc:2454] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4edbdab4425a4865ac47e72440245d75 in term 1.
I20260808 00:37:43.479416 10400 leader_election.cc:302] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4edbdab4425a4865ac47e72440245d75, a0e1fb3ee4404de9871b2f251bced24e; no voters:
I20260808 00:37:43.479504 10646 raft_consensus.cc:2781] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:43.479663 10646 raft_consensus.cc:686] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 1 LEADER]: Becoming Leader. State: Replica: 4edbdab4425a4865ac47e72440245d75, State: Running, Role: LEADER
I20260808 00:37:43.479784 10646 consensus_queue.cc:237] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } }
I20260808 00:37:43.480569 9771 catalog_manager.cc:5495] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4edbdab4425a4865ac47e72440245d75 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "4edbdab4425a4865ac47e72440245d75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4edbdab4425a4865ac47e72440245d75" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 45457 } health_report { overall_health: HEALTHY } } }
I20260808 00:37:43.487596 10618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.486804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53206 (local address 127.1.208.3:45457)
0808 00:37:43.486843 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.486845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.486853 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:43.486930 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.486931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.486942 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.486987 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.487467 (+ 480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.487468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.487468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.487470 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.487480 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.487482 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.487511 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.487545 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.487554 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:37:43.489260 10044 raft_consensus.cc:1261] T 5217a4a85344464b93a2344969c4252a P 5a7d1874f54f48e4acf83625779fb422 [term 1 FOLLOWER]: Refusing update from remote peer 4edbdab4425a4865ac47e72440245d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:43.489260 10280 raft_consensus.cc:1261] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 1 FOLLOWER]: Refusing update from remote peer 4edbdab4425a4865ac47e72440245d75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:43.489591 10646 consensus_queue.cc:1035] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5a7d1874f54f48e4acf83625779fb422" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 36173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:43.489740 10655 consensus_queue.cc:1035] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0e1fb3ee4404de9871b2f251bced24e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:43.492575 10661 mvcc.cc:205] Tried to move back new op lower bound from 7316068202449596416 to 7316068202413854720. Current Snapshot: MvccSnapshot[applied={T|T < 7316068202449596416}]
I20260808 00:37:43.494364 9771 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:43.494444 9771 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:39944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:43.495110 9771 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4178ca105f8d44e79a1b67c7cdd3e571: Sending DeleteTablet for 3 replicas of tablet 5217a4a85344464b93a2344969c4252a
I20260808 00:37:43.495424 10007 tablet_service.cc:1514] Processing DeleteTablet for tablet 5217a4a85344464b93a2344969c4252a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:43 UTC) from {username='slave'} at 127.0.0.1:51010
I20260808 00:37:43.495431 10244 tablet_service.cc:1514] Processing DeleteTablet for tablet 5217a4a85344464b93a2344969c4252a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:43 UTC) from {username='slave'} at 127.0.0.1:48546
I20260808 00:37:43.495659 10675 tablet_replica.cc:330] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: stopping tablet replica
I20260808 00:37:43.495641 10477 tablet_service.cc:1514] Processing DeleteTablet for tablet 5217a4a85344464b93a2344969c4252a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:43 UTC) from {username='slave'} at 127.0.0.1:53202
I20260808 00:37:43.495814 10675 raft_consensus.cc:2229] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:43.495867 10677 tablet_replica.cc:330] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: stopping tablet replica
I20260808 00:37:43.495966 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 9877
I20260808 00:37:43.495976 10677 raft_consensus.cc:2229] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:43.496062 10677 raft_consensus.cc:2258] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:43.496083 10675 raft_consensus.cc:2258] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:43.496919 10677 ts_tablet_manager.cc:1905] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:43.497148 10675 ts_tablet_manager.cc:1905] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:43.498627 10677 ts_tablet_manager.cc:1918] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:43.498706 10677 log.cc:1196] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/5217a4a85344464b93a2344969c4252a
I20260808 00:37:43.499055 10677 ts_tablet_manager.cc:1939] T 5217a4a85344464b93a2344969c4252a P 4edbdab4425a4865ac47e72440245d75: Deleting consensus metadata
I20260808 00:37:43.499353 10675 ts_tablet_manager.cc:1918] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:43.499496 10675 log.cc:1196] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/5217a4a85344464b93a2344969c4252a
I20260808 00:37:43.499516 9744 catalog_manager.cc:4841] TS 4edbdab4425a4865ac47e72440245d75 (127.1.208.3:45457): tablet 5217a4a85344464b93a2344969c4252a (table test-table [id=a93f80a4b4c24071a061b1ccb0858b12]) successfully deleted
I20260808 00:37:43.499876 10675 ts_tablet_manager.cc:1939] T 5217a4a85344464b93a2344969c4252a P a0e1fb3ee4404de9871b2f251bced24e: Deleting consensus metadata
I20260808 00:37:43.501948 9741 catalog_manager.cc:4841] TS a0e1fb3ee4404de9871b2f251bced24e (127.1.208.2:38255): tablet 5217a4a85344464b93a2344969c4252a (table test-table [id=a93f80a4b4c24071a061b1ccb0858b12]) successfully deleted
W20260808 00:37:43.513356 9744 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:36173 (error 108)
I20260808 00:37:43.513643 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 10132
I20260808 00:37:43.521781 10617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.513785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:36173 (local address 127.0.0.1:51016)
0808 00:37:43.521677 (+ 7892us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.521743 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:36173: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7861}
W20260808 00:37:43.521947 9744 catalog_manager.cc:4584] TS 5a7d1874f54f48e4acf83625779fb422 (127.1.208.1:36173): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5217a4a85344464b93a2344969c4252a: Network error: Client connection negotiation failed: client connection to 127.1.208.1:36173: connect: Connection refused (error 111)
I20260808 00:37:43.530612 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 10372
I20260808 00:37:43.544958 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 9717
2026-08-08T00:37:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (688 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 00:37:43.568396 1856 mini_oidc.cc:152] Starting JWKS server
I20260808 00:37:43.569730 1856 webserver.cc:460] Webserver started at https://127.0.0.1:41959/ using document root <none> and password file <none>
I20260808 00:37:43.569828 1856 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:37:43.570031 1856 webserver.cc:460] Webserver started at http://127.0.0.1:32897/ using document root <none> and password file <none>
2026-08-08T00:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:43Z Disabled control of system clock
I20260808 00:37:43.579048 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45903
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:39511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:45903
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41959/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/testchainca.cert with env {}
W20260808 00:37:43.661267 10691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:43.661499 10691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:43.661528 10691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:43.662662 10691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:43.662703 10691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:43.662714 10691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:43.662725 10691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:43.662739 10691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:37:43.662750 10691 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:37:43.662768 10691 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41959/jwks.json
I20260808 00:37:43.664325 10691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:45903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41959/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:43.664537 10691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:43.664705 10691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:43.669128 10700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.669206 10704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.669149 10699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.669390 10691 server_base.cc:1034] running on GCE node
I20260808 00:37:43.669611 10691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:43.669866 10691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:43.670997 10691 hybrid_clock.cc:648] HybridClock initialized: now 1786149463670987 us; error 28 us; skew 500 ppm
I20260808 00:37:43.672185 10691 webserver.cc:460] Webserver started at http://127.1.208.62:42109/ using document root <none> and password file <none>
I20260808 00:37:43.672413 10691 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:43.672461 10691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:43.672586 10691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:43.673732 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "f82842c97be9448aa852c4394452c195"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "26a29635560e9d2d5b8b2950dc5e4eb0"
server_key_iv: "b84415eb5f4ab88feae6fb0ee1eb3a7c"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.674124 10691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "f82842c97be9448aa852c4394452c195"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "26a29635560e9d2d5b8b2950dc5e4eb0"
server_key_iv: "b84415eb5f4ab88feae6fb0ee1eb3a7c"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.675427 10691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:43.676111 10712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.676278 10691 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:43.676364 10691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "f82842c97be9448aa852c4394452c195"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "26a29635560e9d2d5b8b2950dc5e4eb0"
server_key_iv: "b84415eb5f4ab88feae6fb0ee1eb3a7c"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.676429 10691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:43.692453 10691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:43.692775 10691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:43.692914 10691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:43.697171 10691 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:45903
I20260808 00:37:43.697206 10810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:45903 every 8 connection(s)
I20260808 00:37:43.697566 10691 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:43.698511 10811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:43.700850 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: Bootstrap starting.
I20260808 00:37:43.701475 10811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:43.701751 10811 log.cc:824] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:43.702386 10811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: No bootstrap required, opened a new log
I20260808 00:37:43.703450 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 10691
I20260808 00:37:43.703542 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:43.703715 1856 external_mini_cluster.cc:1428] Setting key 0c88bc1f7c24b70771a1037af674649a
I20260808 00:37:43.704216 10811 raft_consensus.cc:348] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } }
I20260808 00:37:43.704406 10811 raft_consensus.cc:374] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:43.704445 10811 raft_consensus.cc:729] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f82842c97be9448aa852c4394452c195, State: Initialized, Role: FOLLOWER
I20260808 00:37:43.704574 10811 consensus_queue.cc:260] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } }
I20260808 00:37:43.704650 10811 raft_consensus.cc:388] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:43.704682 10811 raft_consensus.cc:482] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:43.704742 10811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:43.705411 10811 raft_consensus.cc:504] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } }
I20260808 00:37:43.705515 10811 leader_election.cc:302] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f82842c97be9448aa852c4394452c195; no voters:
I20260808 00:37:43.705665 10811 leader_election.cc:288] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:43.705705 10817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:43.705862 10817 raft_consensus.cc:686] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [term 1 LEADER]: Becoming Leader. State: Replica: f82842c97be9448aa852c4394452c195, State: Running, Role: LEADER
I20260808 00:37:43.705924 10811 sys_catalog.cc:564] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:43.705953 10817 consensus_queue.cc:237] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } }
I20260808 00:37:43.706313 10817 sys_catalog.cc:455] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f82842c97be9448aa852c4394452c195. Latest consensus state: current_term: 1 leader_uuid: "f82842c97be9448aa852c4394452c195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } } }
I20260808 00:37:43.706393 10817 sys_catalog.cc:458] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:43.706329 10823 sys_catalog.cc:455] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f82842c97be9448aa852c4394452c195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f82842c97be9448aa852c4394452c195" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45903 } } }
I20260808 00:37:43.706549 10823 sys_catalog.cc:458] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195 [sys.catalog]: This master's current role is: LEADER
W20260808 00:37:43.707860 10846 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:43.707957 10846 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:43.708076 10831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:43.708277 10831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:43.709861 10831 catalog_manager.cc:1347] Generated new cluster ID: ae2d56e0b42748e4b8df99cc8a84c8fa
I20260808 00:37:43.709921 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.704306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37356 (local address 127.1.208.62:45903)
0808 00:37:43.704549 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.704558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.704592 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:37:43.704690 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.704698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.704770 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.704904 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.705145 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.706756 (+ 1611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.707793 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.708328 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.708447 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:43.709112 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.709116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.709116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.709286 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.709353 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.709368 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.709426 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.709629 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.709706 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":90,"threads_started":1}
I20260808 00:37:43.709924 10831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:43.710820 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:43.718834 10831 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:43.719511 10831 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:43.724463 10831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f82842c97be9448aa852c4394452c195: Generated new TSK 0
I20260808 00:37:43.724627 10831 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:37:43.747555 10863 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 00:37:43.747695 10863 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:37:43.747895 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.732668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.1.208.62:45903)
0808 00:37:43.732713 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.732716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.732866 (+ 150us) server_negotiation.cc:407] Connection header received
0808 00:37:43.732974 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.732975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.732985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.733039 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:37:43.733203 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.733737 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.734899 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.735070 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.735111 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:43.735311 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:37:43.747687 (+ 12376us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:37:43.747841 (+ 154us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37360: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:37:43.748067 10815 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37360: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 00:37:43.748373 10862 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:43.732577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45903 (local address 127.0.0.1:37360)
0808 00:37:43.732778 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:43.732789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:43.732881 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:43.733045 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:43.733046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:43.733047 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:37:43.733120 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:43.733123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.733841 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.733842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:43.734814 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:43.734818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.735127 (+ 309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.735128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:43.735195 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:43.735219 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:37:43.747885 (+ 12666us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 00:37:43.748013 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.208.62:45903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.208.62:45903: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41959/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 00:37:43.748783 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 10691
2026-08-08T00:37:43Z chronyd exiting
I20260808 00:37:43.799367 1856 test_util.cc:175] -----------------------------------------------
I20260808 00:37:43.799455 1856 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786149440113891-1856-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 00:37:43.800300 1856 mini_oidc.cc:152] Starting JWKS server
I20260808 00:37:43.800971 1856 webserver.cc:460] Webserver started at https://127.0.0.1:44003/ using document root <none> and password file <none>
I20260808 00:37:43.801013 1856 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:37:43.801162 1856 webserver.cc:460] Webserver started at http://127.0.0.1:39101/ using document root <none> and password file <none>
2026-08-08T00:37:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:43Z Disabled control of system clock
I20260808 00:37:43.809985 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:35775
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:40429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:35775
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44003/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/testchainca.cert with env {}
W20260808 00:37:43.898438 10869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:43.898655 10869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:43.898677 10869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:43.899804 10869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:43.899838 10869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:43.899848 10869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:43.899860 10869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:43.899876 10869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:37:43.899892 10869 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:37:43.899906 10869 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44003/jwks.json
I20260808 00:37:43.901552 10869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:40429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:35775
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:35775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44003/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:43.901806 10869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:43.901990 10869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:43.906803 10881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.906934 10869 server_base.cc:1034] running on GCE node
W20260808 00:37:43.906806 10877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:43.907047 10879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:43.907305 10869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:43.907564 10869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:43.908699 10869 hybrid_clock.cc:648] HybridClock initialized: now 1786149463908688 us; error 41 us; skew 500 ppm
I20260808 00:37:43.909899 10869 webserver.cc:460] Webserver started at http://127.1.208.62:40427/ using document root <none> and password file <none>
I20260808 00:37:43.910143 10869 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:43.910188 10869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:43.910315 10869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:43.911494 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "0089a3ef44a94081a99504bc03b32684"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "bc101b302f692c0d19fe9f99cbf57d8b"
server_key_iv: "5638e25a269446877b87f722fc2ee4c7"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.911846 10869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "0089a3ef44a94081a99504bc03b32684"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "bc101b302f692c0d19fe9f99cbf57d8b"
server_key_iv: "5638e25a269446877b87f722fc2ee4c7"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.913199 10869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:43.913911 10889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.914052 10869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:43.914139 10869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "0089a3ef44a94081a99504bc03b32684"
format_stamp: "Formatted at 2026-08-08 00:37:43 on dist-test-slave-gjw7"
server_key: "bc101b302f692c0d19fe9f99cbf57d8b"
server_key_iv: "5638e25a269446877b87f722fc2ee4c7"
server_key_version: "encryptionkey@0"
I20260808 00:37:43.914206 10869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:43.930600 10869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:43.930938 10869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:43.931084 10869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:43.935096 10869 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:35775
I20260808 00:37:43.935124 10976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:35775 every 8 connection(s)
I20260808 00:37:43.935458 10869 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:43.936414 10981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:43.938890 10981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684: Bootstrap starting.
I20260808 00:37:43.939496 10981 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:43.939756 10981 log.cc:824] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:43.940435 10981 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684: No bootstrap required, opened a new log
I20260808 00:37:43.942111 10981 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } }
I20260808 00:37:43.942253 10981 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:43.942294 10981 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0089a3ef44a94081a99504bc03b32684, State: Initialized, Role: FOLLOWER
I20260808 00:37:43.942443 10981 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } }
I20260808 00:37:43.942525 10981 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:43.942559 10981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:43.942606 10981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:43.944608 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 10869
I20260808 00:37:43.944706 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:43.944851 1856 external_mini_cluster.cc:1428] Setting key 963a311a0543062733d4b5b3e1df57a1
I20260808 00:37:43.949596 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:43.945320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.1.208.62:35775)
0808 00:37:43.945533 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:43.945541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:43.945567 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:37:43.945664 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:43.945669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:43.945721 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:43.945859 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:43.946065 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.946952 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.947699 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:43.948108 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:43.948219 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:43.948850 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:43.948854 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:43.948854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:43.949017 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:43.949050 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:43.949062 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:43.949118 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:43.949203 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 00:37:43.949286 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260808 00:37:43.950462 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:43.962347 10981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } }
I20260808 00:37:43.962505 10981 leader_election.cc:302] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0089a3ef44a94081a99504bc03b32684; no voters:
I20260808 00:37:43.962687 10981 leader_election.cc:288] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:43.962778 10988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:43.962949 10981 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:43.962980 10988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [term 1 LEADER]: Becoming Leader. State: Replica: 0089a3ef44a94081a99504bc03b32684, State: Running, Role: LEADER
I20260808 00:37:43.963078 10988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } }
I20260808 00:37:43.963478 10989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0089a3ef44a94081a99504bc03b32684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } } }
I20260808 00:37:43.963564 10989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:43.963780 10990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0089a3ef44a94081a99504bc03b32684. Latest consensus state: current_term: 1 leader_uuid: "0089a3ef44a94081a99504bc03b32684" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0089a3ef44a94081a99504bc03b32684" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 35775 } } }
I20260808 00:37:43.964134 10990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:43.964231 11003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:43.964802 11003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:43.966528 11003 catalog_manager.cc:1347] Generated new cluster ID: 89239be72351430a98582143874d14a3
I20260808 00:37:43.966583 11003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:43.987052 11003 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:43.987572 11003 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:43.992292 11003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0089a3ef44a94081a99504bc03b32684: Generated new TSK 0
I20260808 00:37:43.992475 11003 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 00:37:44.020375 11033 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 00:37:44.020452 11033 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 00:37:44.020808 10987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:44.006433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.1.208.62:35775)
0808 00:37:44.006480 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:44.006483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:44.006568 (+ 85us) server_negotiation.cc:407] Connection header received
0808 00:37:44.006636 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:44.006636 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:44.006648 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:44.006718 (+ 70us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:37:44.006823 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.007625 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.008360 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.008687 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.008756 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.008886 (+ 130us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:37:44.020578 (+ 11692us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:37:44.020733 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:37:44.020859 11032 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:44.006364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:35775 (local address 127.0.0.1:40752)
0808 00:37:44.006499 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:44.006509 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:44.006564 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:44.006717 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:44.006718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:44.006719 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:37:44.006771 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.006773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.007690 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.007691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.008279 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.008281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.008754 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.008755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.008792 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.008797 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:37:44.020684 (+ 11887us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 00:37:44.020780 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.208.62:35775: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
W20260808 00:37:44.020882 10987 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40752: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44003/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260808 00:37:44.021225 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 10869
2026-08-08T00:37:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (272 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 00:37:44.072944 1856 mini_oidc.cc:152] Starting JWKS server
I20260808 00:37:44.073673 1856 webserver.cc:460] Webserver started at https://127.0.0.1:38909/ using document root <none> and password file <none>
I20260808 00:37:44.073719 1856 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 00:37:44.073868 1856 webserver.cc:460] Webserver started at http://127.0.0.1:46527/ using document root <none> and password file <none>
2026-08-08T00:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:44Z Disabled control of system clock
I20260808 00:37:44.083316 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38839
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:42003
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38839
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38909/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:37:44.170840 11048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:44.171082 11048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:44.171106 11048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:44.172295 11048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:44.172343 11048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:44.172362 11048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:44.172375 11048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:44.172389 11048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:37:44.172411 11048 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:37:44.172439 11048 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38909/jwks.json
I20260808 00:37:44.173934 11048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:42003
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38839
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38909/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:44.174197 11048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:44.174496 11048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:44.179374 11054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:44.179386 11055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:44.179391 11048 server_base.cc:1034] running on GCE node
W20260808 00:37:44.179373 11058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:44.179852 11048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:44.180106 11048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:44.181241 11048 hybrid_clock.cc:648] HybridClock initialized: now 1786149464181224 us; error 34 us; skew 500 ppm
I20260808 00:37:44.182428 11048 webserver.cc:460] Webserver started at http://127.1.208.62:35321/ using document root <none> and password file <none>
I20260808 00:37:44.182614 11048 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:44.182653 11048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:44.182745 11048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:44.183861 11048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "3ab61a87ff8247d998912070cf2080a4"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "c68b51e953418155055f15ff9766b47d"
server_key_iv: "2551c1723646a08e468764f74dc8fd6b"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.184171 11048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "3ab61a87ff8247d998912070cf2080a4"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "c68b51e953418155055f15ff9766b47d"
server_key_iv: "2551c1723646a08e468764f74dc8fd6b"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.185406 11048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:44.186072 11068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:44.186221 11048 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:44.186307 11048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "3ab61a87ff8247d998912070cf2080a4"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "c68b51e953418155055f15ff9766b47d"
server_key_iv: "2551c1723646a08e468764f74dc8fd6b"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.186373 11048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:44.201432 11048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:44.201740 11048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:44.201879 11048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:44.205964 11048 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38839
I20260808 00:37:44.205998 11150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38839 every 8 connection(s)
I20260808 00:37:44.206329 11048 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:44.206991 11151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:44.207621 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 11048
I20260808 00:37:44.207736 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:44.207952 1856 external_mini_cluster.cc:1428] Setting key eca17bc3796bab7f2f753fd5bd4c9e57
I20260808 00:37:44.210709 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: Bootstrap starting.
I20260808 00:37:44.211294 11151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:44.211555 11151 log.cc:824] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:44.212182 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: No bootstrap required, opened a new log
I20260808 00:37:44.213281 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:44.208555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.1.208.62:38839)
0808 00:37:44.208765 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:44.208773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:44.208798 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:37:44.209256 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:44.209262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:44.209371 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:44.209497 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:44.209704 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.210610 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.211464 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.211906 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.212017 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.212695 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:44.212699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:44.212699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:44.212855 (+ 156us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:44.212888 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:44.212898 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:44.212951 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:44.213038 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 00:37:44.213119 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260808 00:37:44.214226 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:44.214125 11151 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } }
I20260808 00:37:44.214283 11151 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:44.214330 11151 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ab61a87ff8247d998912070cf2080a4, State: Initialized, Role: FOLLOWER
I20260808 00:37:44.214453 11151 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } }
I20260808 00:37:44.214515 11151 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:44.214545 11151 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:44.214592 11151 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:44.271052 11151 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } }
I20260808 00:37:44.271273 11151 leader_election.cc:302] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ab61a87ff8247d998912070cf2080a4; no voters:
I20260808 00:37:44.271557 11151 leader_election.cc:288] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:44.271660 11158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:44.271834 11158 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [term 1 LEADER]: Becoming Leader. State: Replica: 3ab61a87ff8247d998912070cf2080a4, State: Running, Role: LEADER
I20260808 00:37:44.271886 11151 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:44.271960 11158 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } }
I20260808 00:37:44.272375 11160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ab61a87ff8247d998912070cf2080a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } } }
I20260808 00:37:44.272464 11160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:44.272692 11161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ab61a87ff8247d998912070cf2080a4. Latest consensus state: current_term: 1 leader_uuid: "3ab61a87ff8247d998912070cf2080a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ab61a87ff8247d998912070cf2080a4" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38839 } } }
I20260808 00:37:44.272811 11161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4 [sys.catalog]: This master's current role is: LEADER
W20260808 00:37:44.273810 11189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:44.273866 11189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:44.273968 11177 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:44.274120 11177 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:44.278141 11177 catalog_manager.cc:1347] Generated new cluster ID: d707ef47fb3641b2b3742cbcb877fd82
I20260808 00:37:44.278209 11177 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:44.293891 11177 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:44.294600 11177 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:44.299662 11177 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ab61a87ff8247d998912070cf2080a4: Generated new TSK 0
I20260808 00:37:44.299861 11177 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:44.309211 11155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:44.305556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59114 (local address 127.1.208.62:38839)
0808 00:37:44.305614 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:44.305616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:44.305681 (+ 65us) server_negotiation.cc:407] Connection header received
0808 00:37:44.305738 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:44.305740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:44.305754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:44.305849 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:37:44.305956 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.306760 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.307511 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.307815 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.307893 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.308031 (+ 138us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:37:44.308996 (+ 965us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38909/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:37:44.309130 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38909/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":33}
W20260808 00:37:44.309271 11155 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38909/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 00:37:44.311156 11204 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:44.305456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38839 (local address 127.0.0.1:59114)
0808 00:37:44.305622 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:44.305631 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:44.305683 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:44.305848 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:44.305848 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:44.305850 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:37:44.305899 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.305901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.306840 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.306841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.307438 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.307440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.307892 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.307892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.307943 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.307948 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:37:44.309099 (+ 1151us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38909/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 00:37:44.311056 (+ 1957us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.208.62:38839: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38909/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":1761,"thread_start_us":96,"threads_started":1}
I20260808 00:37:44.311478 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 11048
E20260808 00:37:44.326511 11205 webserver.cc:533] Webserver: sslize failed:
E20260808 00:37:44.326583 11205 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T00:37:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (284 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T00:37:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:44Z Disabled control of system clock
I20260808 00:37:44.365800 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34745
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:37413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34745
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/testchainca.cert with env {}
W20260808 00:37:44.448773 11211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:44.448976 11211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:44.448997 11211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:44.450320 11211 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:44.450376 11211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:44.450394 11211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:44.450407 11211 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:44.450418 11211 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 00:37:44.450429 11211 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 00:37:44.450439 11211 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 00:37:44.451887 11211 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34745
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11211
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:44.452086 11211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:44.452261 11211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:44.456943 11221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:44.457007 11211 server_base.cc:1034] running on GCE node
W20260808 00:37:44.456954 11227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:44.456960 11219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:44.457404 11211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:44.457690 11211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:44.458853 11211 hybrid_clock.cc:648] HybridClock initialized: now 1786149464458844 us; error 36 us; skew 500 ppm
I20260808 00:37:44.460094 11211 webserver.cc:460] Webserver started at http://127.1.208.62:37953/ using document root <none> and password file <none>
I20260808 00:37:44.460309 11211 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:44.460357 11211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:44.460484 11211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:44.461591 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "5c41a5b83c1d4f968b915861bd0adbf8"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "bab0370eb5c833d0dc4936d7b0b50429"
server_key_iv: "34eeb316214a4ea8d1e585ac162d17b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.461968 11211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "5c41a5b83c1d4f968b915861bd0adbf8"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "bab0370eb5c833d0dc4936d7b0b50429"
server_key_iv: "34eeb316214a4ea8d1e585ac162d17b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.463546 11211 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:44.464258 11238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:44.464411 11211 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:44.464494 11211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "5c41a5b83c1d4f968b915861bd0adbf8"
format_stamp: "Formatted at 2026-08-08 00:37:44 on dist-test-slave-gjw7"
server_key: "bab0370eb5c833d0dc4936d7b0b50429"
server_key_iv: "34eeb316214a4ea8d1e585ac162d17b9"
server_key_version: "encryptionkey@0"
I20260808 00:37:44.464557 11211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:44.477934 11211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:44.478217 11211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:44.478338 11211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:44.482492 11211 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34745
I20260808 00:37:44.482524 11323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34745 every 8 connection(s)
I20260808 00:37:44.482815 11211 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:44.483723 11324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:44.486024 11324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8: Bootstrap starting.
I20260808 00:37:44.486600 11324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:44.486856 11324 log.cc:824] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:44.487490 11324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8: No bootstrap required, opened a new log
I20260808 00:37:44.489075 11324 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } }
I20260808 00:37:44.489209 11324 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:44.489241 11324 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c41a5b83c1d4f968b915861bd0adbf8, State: Initialized, Role: FOLLOWER
I20260808 00:37:44.489392 11324 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } }
I20260808 00:37:44.489466 11324 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:44.489490 11324 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:44.489533 11324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:44.490077 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 11211
I20260808 00:37:44.490178 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:44.490352 1856 external_mini_cluster.cc:1428] Setting key 909a1d249fe219faf6631cfd9a9f2e03
I20260808 00:37:44.494808 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:44.490726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42818 (local address 127.1.208.62:34745)
0808 00:37:44.490927 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:44.490935 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:44.490967 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:37:44.491060 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:44.491065 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:44.491114 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:44.491223 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:44.491428 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.492269 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.493026 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.493463 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.493573 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.494136 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:44.494141 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:44.494141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:37:44.494307 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:44.494343 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:37:44.494353 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:44.494414 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:44.494481 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 00:37:44.494560 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":72,"threads_started":1}
I20260808 00:37:44.495641 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:37:44.581207 11324 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } }
I20260808 00:37:44.581460 11324 leader_election.cc:302] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c41a5b83c1d4f968b915861bd0adbf8; no voters:
I20260808 00:37:44.581708 11324 leader_election.cc:288] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:44.581785 11332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:44.581985 11332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [term 1 LEADER]: Becoming Leader. State: Replica: 5c41a5b83c1d4f968b915861bd0adbf8, State: Running, Role: LEADER
I20260808 00:37:44.582033 11324 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:44.582094 11332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } }
I20260808 00:37:44.582504 11332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c41a5b83c1d4f968b915861bd0adbf8. Latest consensus state: current_term: 1 leader_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } } }
I20260808 00:37:44.582580 11332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:44.583308 11333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c41a5b83c1d4f968b915861bd0adbf8" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34745 } } }
I20260808 00:37:44.583392 11333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:44.583604 11341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:44.584391 11341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:44.586119 11341 catalog_manager.cc:1347] Generated new cluster ID: 6f77551aeaf9478eb1b4c57e4ba98b1a
I20260808 00:37:44.586164 11341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:44.595527 11341 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:44.596233 11341 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:44.608475 11341 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c41a5b83c1d4f968b915861bd0adbf8: Generated new TSK 0
I20260808 00:37:44.608675 11341 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:45.042095 11330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:44.613358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42832 (local address 127.1.208.62:34745)
0808 00:37:44.613405 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:44.613408 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:44.613492 (+ 84us) server_negotiation.cc:407] Connection header received
0808 00:37:44.613560 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:44.613561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:44.613574 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:44.613638 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0808 00:37:44.613765 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.614288 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.615095 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.615316 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.615376 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.615533 (+ 157us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 00:37:45.041762 (+426229us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:37:45.041989 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42832: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 00:37:45.042183 11369 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:44.613263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34745 (local address 127.0.0.1:42832)
0808 00:37:44.613426 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:44.613435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:44.613496 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:44.613638 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:44.613639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:44.613641 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0808 00:37:44.613704 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.613707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.614389 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.614390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.615012 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:44.615015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:44.615389 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:44.615390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:44.615443 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:44.615448 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 00:37:45.041953 (+426505us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 00:37:45.042083 (+ 130us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.208.62:34745: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
W20260808 00:37:45.042196 11330 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42832: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260808 00:37:45.042491 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 11211
2026-08-08T00:37:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (715 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (97 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (96 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:45 dist-test-slave-gjw7 krb5kdc[11391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:45 dist-test-slave-gjw7 krb5kdc[11391](info): set up 2 sockets
Aug 08 00:37:45 dist-test-slave-gjw7 krb5kdc[11391](info): commencing operation
krb5kdc: starting...
W20260808 00:37:46.830622 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.564s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:46 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:46.949745 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42663
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42663
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:47.031450 11416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:47.031641 11416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:47.031661 11416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:47.032776 11416 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:47.032816 11416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:47.032828 11416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:47.032840 11416 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:47.032862 11416 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:47.034657 11416 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:47.034920 11416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:47.035142 11416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:47.039968 11429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.040016 11416 server_base.cc:1034] running on GCE node
W20260808 00:37:47.039979 11427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:47.039981 11426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.040475 11416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:47.040753 11416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:47.041880 11416 hybrid_clock.cc:648] HybridClock initialized: now 1786149467041864 us; error 34 us; skew 500 ppm
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.044363 11416 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:37:47.044796 11416 webserver.cc:460] Webserver started at http://127.1.208.62:37091/ using document root <none> and password file <none>
I20260808 00:37:47.045022 11416 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:47.045068 11416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:47.045198 11416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:47.046290 11416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "db6ab1bc5b414201a4d37dfe25d1aee0"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "cdb4f2b4d5417b69cd44a5187061314d"
server_key_iv: "61c4d2b3ccfd47b27a15423c3ebac67a"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.046624 11416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "db6ab1bc5b414201a4d37dfe25d1aee0"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "cdb4f2b4d5417b69cd44a5187061314d"
server_key_iv: "61c4d2b3ccfd47b27a15423c3ebac67a"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.047924 11416 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:37:47.048609 11439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.048801 11416 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:47.048895 11416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "db6ab1bc5b414201a4d37dfe25d1aee0"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "cdb4f2b4d5417b69cd44a5187061314d"
server_key_iv: "61c4d2b3ccfd47b27a15423c3ebac67a"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.048951 11416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:47.058123 11416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:47.060190 11416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:47.060305 11416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:47.064214 11416 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:42663
I20260808 00:37:47.064267 11526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:42663 every 8 connection(s)
I20260808 00:37:47.064575 11416 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:47.065485 11530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:47.067783 11530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0: Bootstrap starting.
I20260808 00:37:47.068361 11530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:47.068627 11530 log.cc:824] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:47.069254 11530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0: No bootstrap required, opened a new log
I20260808 00:37:47.070844 11530 raft_consensus.cc:348] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } }
I20260808 00:37:47.070978 11530 raft_consensus.cc:374] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:47.071008 11530 raft_consensus.cc:729] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db6ab1bc5b414201a4d37dfe25d1aee0, State: Initialized, Role: FOLLOWER
I20260808 00:37:47.071141 11530 consensus_queue.cc:260] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } }
I20260808 00:37:47.071198 11530 raft_consensus.cc:388] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:47.071219 11530 raft_consensus.cc:482] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:47.071262 11530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:47.071739 11530 raft_consensus.cc:504] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } }
I20260808 00:37:47.071838 11530 leader_election.cc:302] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db6ab1bc5b414201a4d37dfe25d1aee0; no voters:
I20260808 00:37:47.071990 11530 leader_election.cc:288] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:47.072029 11536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:47.072229 11530 sys_catalog.cc:564] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:47.072338 11536 raft_consensus.cc:686] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [term 1 LEADER]: Becoming Leader. State: Replica: db6ab1bc5b414201a4d37dfe25d1aee0, State: Running, Role: LEADER
I20260808 00:37:47.072458 11536 consensus_queue.cc:237] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } }
I20260808 00:37:47.072794 11540 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader db6ab1bc5b414201a4d37dfe25d1aee0. Latest consensus state: current_term: 1 leader_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } } }
I20260808 00:37:47.072804 11538 sys_catalog.cc:455] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db6ab1bc5b414201a4d37dfe25d1aee0" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42663 } } }
I20260808 00:37:47.072958 11540 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:47.072970 11538 sys_catalog.cc:458] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:47.073347 11547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:47.073799 11547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:47.074132 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 11416
I20260808 00:37:47.074234 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:47.074435 1856 external_mini_cluster.cc:1428] Setting key e79ed89eff6b5143e76e8f325a4b1b67
I20260808 00:37:47.075757 11547 catalog_manager.cc:1347] Generated new cluster ID: 48e089641e1d414a8b7209fa8aa948d8
I20260808 00:37:47.075804 11547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:47.088937 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.074821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.1.208.62:42663)
0808 00:37:47.075123 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.075131 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.075155 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:37:47.075253 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.075259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.075300 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.075397 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.076252 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.077683 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.078735 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.079324 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.079419 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.085170 (+ 5751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.085175 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.085176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.085191 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.087109 (+ 1918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.087523 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.087526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.087527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.087644 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.087835 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.087835 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.087836 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.088167 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.088303 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.088405 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.088666 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":77,"threads_started":1}
I20260808 00:37:47.096511 11547 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:47.097007 11547 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:47.107915 11547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db6ab1bc5b414201a4d37dfe25d1aee0: Generated new TSK 0
I20260808 00:37:47.108094 11547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:37:47.140856 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:47.240499 11576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:47.240696 11576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:47.240718 11576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:47.241878 11576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:47.241928 11576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:47.241955 11576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:47.243446 11576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:47.243672 11576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:47.243885 11576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:47.248744 11585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:47.248807 11588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.248905 11576 server_base.cc:1034] running on GCE node
W20260808 00:37:47.248752 11586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.249181 11576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:47.249449 11576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:47.250592 11576 hybrid_clock.cc:648] HybridClock initialized: now 1786149467250573 us; error 36 us; skew 500 ppm
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.253224 11576 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:37:47.253647 11576 webserver.cc:460] Webserver started at http://127.1.208.1:36425/ using document root <none> and password file <none>
I20260808 00:37:47.253881 11576 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:47.253931 11576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:47.254062 11576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:47.255139 11576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "50ce5454ac0f44c7b4b673886b72e117"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "57bf7b78f8b6acd1641440adc7e0c6dd"
server_key_iv: "693f1369ae7f2a3dce08ba2cac9da396"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.255479 11576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "50ce5454ac0f44c7b4b673886b72e117"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "57bf7b78f8b6acd1641440adc7e0c6dd"
server_key_iv: "693f1369ae7f2a3dce08ba2cac9da396"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.256745 11576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:47.257459 11606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.257591 11576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:47.257666 11576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "50ce5454ac0f44c7b4b673886b72e117"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "57bf7b78f8b6acd1641440adc7e0c6dd"
server_key_iv: "693f1369ae7f2a3dce08ba2cac9da396"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.257728 11576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:47.273346 11576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:47.275154 11576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:47.275286 11576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:47.275507 11576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:47.275800 11576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:47.275839 11576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.275868 11576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:47.275893 11576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.281589 11576 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:38231
I20260808 00:37:47.281656 11798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:38231 every 8 connection(s)
I20260808 00:37:47.281946 11576 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:47.287047 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 11576
I20260808 00:37:47.287133 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:47.287293 1856 external_mini_cluster.cc:1428] Setting key 7d955152d29c86fb4e3e6a87edcaecf7
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:47.291426 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.282636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:57173 (local address 127.1.208.62:42663)
0808 00:37:47.282715 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.282717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.282902 (+ 185us) server_negotiation.cc:407] Connection header received
0808 00:37:47.283886 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.283887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.283901 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.283952 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.284530 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.285031 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.286069 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.286262 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.286306 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.288478 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.288480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.288480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.288488 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.290013 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.290452 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.290453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.290453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.290503 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.290758 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.290759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.290759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.290863 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.290908 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.291269 (+ 361us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.291356 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:37:47.291591 11804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.282615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42663 (local address 127.1.208.1:57173)
0808 00:37:47.282782 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:47.282820 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:47.283790 (+ 970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:47.284009 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:47.284013 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:47.284275 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:47.284467 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.284471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.285105 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.285105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.285994 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.285998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.286320 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.286321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.286419 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.286832 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:47.286839 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:47.288391 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:47.290087 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.290095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.290097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.290384 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.290553 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.290554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.290554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.290685 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.290915 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:47.290922 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:47.291091 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 00:37:47.291209 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 00:37:47.291376 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":76,"threads_started":1}
I20260808 00:37:47.292146 11802 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42663
I20260808 00:37:47.292285 11802 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:47.292474 11802 heartbeater.cc:511] Master 127.1.208.62:42663 requested a full tablet report, sending...
I20260808 00:37:47.293059 11471 ts_manager.cc:194] Registered new tserver with Master: 50ce5454ac0f44c7b4b673886b72e117 (127.1.208.1:38231)
I20260808 00:37:47.293735 11471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:57173
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:37:47.313987 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:47.397656 11810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:47.397861 11810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:47.397882 11810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:47.398986 11810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:47.399024 11810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:47.399047 11810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:47.400454 11810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:47.400631 11810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:47.400806 11810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:47.405563 11820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.405608 11810 server_base.cc:1034] running on GCE node
W20260808 00:37:47.405576 11819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:47.405578 11822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.406003 11810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:47.406232 11810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:47.407370 11810 hybrid_clock.cc:648] HybridClock initialized: now 1786149467407351 us; error 39 us; skew 500 ppm
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.409834 11810 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:37:47.410221 11810 webserver.cc:460] Webserver started at http://127.1.208.2:43261/ using document root <none> and password file <none>
I20260808 00:37:47.410415 11810 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:47.410463 11810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:47.410588 11810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:47.411592 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "55780a8353df44f0afd2260736162569"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "82b84d71274820031b2cdca33e799a57"
server_key_iv: "6d891710de0eb1281f46899c20274d03"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.411989 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "55780a8353df44f0afd2260736162569"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "82b84d71274820031b2cdca33e799a57"
server_key_iv: "6d891710de0eb1281f46899c20274d03"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.413308 11810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:47.413985 11834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.414142 11810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:47.414235 11810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "55780a8353df44f0afd2260736162569"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "82b84d71274820031b2cdca33e799a57"
server_key_iv: "6d891710de0eb1281f46899c20274d03"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.414299 11810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:47.450943 11810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:47.452961 11810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:47.453088 11810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:47.453372 11810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:47.453686 11810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:47.453718 11810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.453748 11810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:47.453771 11810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.459600 11810 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:38817
I20260808 00:37:47.459673 12034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:38817 every 8 connection(s)
I20260808 00:37:47.459964 11810 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:47.468312 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.460670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:54077 (local address 127.1.208.62:42663)
0808 00:37:47.460752 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.460755 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.460910 (+ 155us) server_negotiation.cc:407] Connection header received
0808 00:37:47.461926 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.461927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.461938 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.461987 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.462564 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.463054 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.463973 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.464148 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.464191 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.466219 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.466221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.466221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.466228 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.467293 (+ 1065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.467636 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.467636 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.467637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.467681 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.467852 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.467853 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.467853 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.467946 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.467989 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.468185 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.468245 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:37:47.468504 12040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.460641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42663 (local address 127.1.208.2:54077)
0808 00:37:47.460793 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:47.460831 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:47.461834 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:47.462046 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:47.462049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:47.462302 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:47.462499 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.462503 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.463113 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.463114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.463912 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.463915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.464201 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.464202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.464296 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.464752 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:47.464760 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:47.466136 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:47.467361 (+ 1225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.467368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.467370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.467581 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.467723 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.467723 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.467723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.467812 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.467985 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:47.467988 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:47.468097 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 00:37:47.468170 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:37:47.468284 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260808 00:37:47.468938 12036 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42663
I20260808 00:37:47.469024 12036 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:47.469190 12036 heartbeater.cc:511] Master 127.1.208.62:42663 requested a full tablet report, sending...
I20260808 00:37:47.469575 11471 ts_manager.cc:194] Registered new tserver with Master: 55780a8353df44f0afd2260736162569 (127.1.208.2:38817)
I20260808 00:37:47.469703 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 11810
I20260808 00:37:47.469798 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:47.469982 1856 external_mini_cluster.cc:1428] Setting key a892675b0d620a293106f6891453b07d
I20260808 00:37:47.470058 11471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:54077
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:37:47.493930 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:47.579499 12043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:47.579720 12043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:47.579746 12043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:47.580874 12043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:47.580920 12043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:47.580952 12043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:47.582515 12043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:47.582757 12043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:47.582973 12043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:47.587952 12056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:47.587996 12061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.587991 12043 server_base.cc:1034] running on GCE node
W20260808 00:37:47.587960 12053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:47.588362 12043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:47.588604 12043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:47.589749 12043 hybrid_clock.cc:648] HybridClock initialized: now 1786149467589731 us; error 33 us; skew 500 ppm
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.592382 12043 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:37:47.592831 12043 webserver.cc:460] Webserver started at http://127.1.208.3:44929/ using document root <none> and password file <none>
I20260808 00:37:47.593037 12043 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:47.593080 12043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:47.593201 12043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:47.594249 12043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "a8f97a2f3cdc4498929324c8ab635b2a"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "d4b3df870590e3a98b96cc7181318a4e"
server_key_iv: "6973498267c1d346ed2d3a101e524a64"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.594584 12043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "a8f97a2f3cdc4498929324c8ab635b2a"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "d4b3df870590e3a98b96cc7181318a4e"
server_key_iv: "6973498267c1d346ed2d3a101e524a64"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.595785 12043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:47.596446 12071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.596583 12043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:47.596659 12043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "a8f97a2f3cdc4498929324c8ab635b2a"
format_stamp: "Formatted at 2026-08-08 00:37:47 on dist-test-slave-gjw7"
server_key: "d4b3df870590e3a98b96cc7181318a4e"
server_key_iv: "6973498267c1d346ed2d3a101e524a64"
server_key_version: "encryptionkey@0"
I20260808 00:37:47.596732 12043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:47.607138 12043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:47.609156 12043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:47.609375 12043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:47.609597 12043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:47.609912 12043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:47.609954 12043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.609977 12043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:47.609999 12043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:47.615867 12043 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:37511
I20260808 00:37:47.615914 12253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:37511 every 8 connection(s)
I20260808 00:37:47.616240 12043 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:47.619454 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 12043
I20260808 00:37:47.619545 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:47.619729 1856 external_mini_cluster.cc:1428] Setting key fe99f5ad2fbac983a1bce65bab1ba064
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:37:47.625159 11571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.616936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:38895 (local address 127.1.208.62:42663)
0808 00:37:47.616979 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.616981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.617187 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:37:47.618261 (+ 1074us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.618262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.618274 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.618332 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.618958 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.619619 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.620801 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.620986 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.621036 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.623055 (+ 2019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.623056 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.623056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.623063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.624122 (+ 1059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.624472 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.624473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.624473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.624526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.624703 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.624703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.624704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.624795 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.624840 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.625038 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.625096 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:47.625345 12259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.616913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42663 (local address 127.1.208.3:38895)
0808 00:37:47.617069 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:47.617112 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:47.618121 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:47.618397 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:47.618400 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:47.618668 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:47.618873 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.618878 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.619700 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.619701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.620737 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:47.620740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.621044 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.621045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:47.621142 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.621591 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:47.621600 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:47.622975 (+ 1375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:47.624189 (+ 1214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.624195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.624197 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.624412 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.624573 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:47.624573 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:47.624573 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:47.624661 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:47.624837 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:47.624840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:47.624945 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 00:37:47.625020 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 00:37:47.625135 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260808 00:37:47.625767 12256 heartbeater.cc:347] Connected to a master server at 127.1.208.62:42663
I20260808 00:37:47.625855 12256 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:47.626024 12256 heartbeater.cc:511] Master 127.1.208.62:42663 requested a full tablet report, sending...
I20260808 00:37:47.626375 11471 ts_manager.cc:194] Registered new tserver with Master: a8f97a2f3cdc4498929324c8ab635b2a (127.1.208.3:37511)
I20260808 00:37:47.626713 11471 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:38895
I20260808 00:37:47.631248 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/test-admin.keytab.
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.647221 1856 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 00:37:47.648830 12272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:47.648097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41938 (local address 127.1.208.1:38231)
0808 00:37:47.648253 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.648258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.648268 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:47.648295 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.648297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.648310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.648376 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.648750 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41938: BlockingRecv error: recv got EOF from 127.0.0.1:41938 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
W20260808 00:37:47.649020 1856 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.649909 1856 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.650673 1856 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:47.657838 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.651141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.1.208.1:38231)
0808 00:37:47.651182 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.651183 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.651220 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:37:47.651299 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.651300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.651310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.651358 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.651718 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.652424 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.653130 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.653499 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.653549 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.655531 (+ 1982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.655539 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.655539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.655547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.656767 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.657073 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.657077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.657077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.657130 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.657312 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.657313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.657314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.657443 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.657576 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.657655 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.657729 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:37:47.659332 12272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:47.658902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.1.208.1:38231)
0808 00:37:47.658961 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.658962 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.658970 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:47.659019 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.659020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.659028 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.659069 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.659286 (+ 217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41964: BlockingRecv error: recv got EOF from 127.0.0.1:41964 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 00:37:47.659468 1856 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.660292 1856 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.661046 1856 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:47.667017 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.661496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.1.208.1:38231)
0808 00:37:47.661531 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.661533 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.661560 (+ 27us) server_negotiation.cc:407] Connection header received
0808 00:37:47.661610 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.661610 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.661619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.661657 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.661941 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.662384 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.662965 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.663138 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.663178 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.664857 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.664858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.664859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.664865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.666086 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.666375 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.666376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.666377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.666427 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.666584 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.666585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.666585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.666674 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.666749 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.666850 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.666931 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 00:37:47.668226 12272 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:47.667850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41978 (local address 127.1.208.1:38231)
0808 00:37:47.667884 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.667885 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.667911 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:37:47.667963 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.667964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.667970 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.668007 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.668187 (+ 180us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41978: BlockingRecv error: recv got EOF from 127.0.0.1:41978 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 00:37:47.668371 1856 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786149440113891-1856-0/krb5kdc/krb5cc' of type FILE
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.669174 1856 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:47.669931 1856 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[11391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:37:47.675784 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:47.670331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41988 (local address 127.1.208.1:38231)
0808 00:37:47.670384 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:47.670385 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:47.670410 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:37:47.670452 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:47.670453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:47.670460 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:47.670500 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:47.670778 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.671233 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.671845 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:47.672031 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:47.672077 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:47.673861 (+ 1784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:47.673862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:47.673862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:47.673868 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:47.675053 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.675301 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.675301 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.675301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.675342 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:47.675455 (+ 113us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:47.675456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:47.675456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:47.675530 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:47.675590 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:47.675656 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 00:37:47.675706 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:37:47.676367 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 11576
I20260808 00:37:47.691192 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 11810
I20260808 00:37:47.702039 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 12043
I20260808 00:37:47.715684 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 11416
2026-08-08T00:37:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2474 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[12292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[12292](info): set up 2 sockets
Aug 08 00:37:47 dist-test-slave-gjw7 krb5kdc[12292](info): commencing operation
krb5kdc: starting...
W20260808 00:37:48.888093 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:48 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab.
Entry for principal oryx/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab.
I20260808 00:37:49.005874 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40611
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab
--principal=oryx/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:49.087215 12313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:49.087446 12313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:49.087481 12313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:49.088622 12313 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:49.088670 12313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:49.088693 12313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:49.088712 12313 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:49.088732 12313 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:49.090257 12313 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab
--principal=oryx/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:49.090518 12313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:49.090739 12313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.095459 12322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.095495 12321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.095527 12328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.095863 12313 server_base.cc:1034] running on GCE node
I20260808 00:37:49.096130 12313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.096411 12313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.097559 12313 hybrid_clock.cc:648] HybridClock initialized: now 1786149469097540 us; error 30 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.099828 12313 init.cc:377] Logged in from keytab as oryx/127.1.208.62@KRBTEST.COM (short username oryx)
I20260808 00:37:49.100276 12313 webserver.cc:460] Webserver started at http://127.1.208.62:38621/ using document root <none> and password file <none>
I20260808 00:37:49.100497 12313 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.100545 12313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.100666 12313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:49.101727 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "c25e8d957afb476aa914c320c4fb374b"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "3996319a772222739583dd56fd7678e1"
server_key_iv: "4ee8612ffa8d3def815fe037c5c3e3d2"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.102059 12313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "c25e8d957afb476aa914c320c4fb374b"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "3996319a772222739583dd56fd7678e1"
server_key_iv: "4ee8612ffa8d3def815fe037c5c3e3d2"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.103278 12313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:49.103940 12340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.104090 12313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:49.104173 12313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "c25e8d957afb476aa914c320c4fb374b"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "3996319a772222739583dd56fd7678e1"
server_key_iv: "4ee8612ffa8d3def815fe037c5c3e3d2"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.104233 12313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.124001 12313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.125833 12313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:49.125973 12313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.130054 12313 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:40611
I20260808 00:37:49.130097 12416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:40611 every 8 connection(s)
I20260808 00:37:49.130462 12313 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:49.131470 12418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:49.133867 12418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Bootstrap starting.
I20260808 00:37:49.134461 12418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:49.134727 12418 log.cc:824] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:49.135380 12418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: No bootstrap required, opened a new log
I20260808 00:37:49.136983 12418 raft_consensus.cc:348] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } }
I20260808 00:37:49.137126 12418 raft_consensus.cc:374] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:49.137157 12418 raft_consensus.cc:729] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25e8d957afb476aa914c320c4fb374b, State: Initialized, Role: FOLLOWER
I20260808 00:37:49.137327 12418 consensus_queue.cc:260] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } }
I20260808 00:37:49.137405 12418 raft_consensus.cc:388] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:49.137441 12418 raft_consensus.cc:482] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:49.137490 12418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:49.138265 12418 raft_consensus.cc:504] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } }
I20260808 00:37:49.138365 12418 leader_election.cc:302] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c25e8d957afb476aa914c320c4fb374b; no voters:
I20260808 00:37:49.138528 12418 leader_election.cc:288] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:49.138561 12422 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:49.138679 12422 raft_consensus.cc:686] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 LEADER]: Becoming Leader. State: Replica: c25e8d957afb476aa914c320c4fb374b, State: Running, Role: LEADER
I20260808 00:37:49.138772 12418 sys_catalog.cc:564] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:49.138796 12422 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } }
I20260808 00:37:49.139207 12425 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } }
I20260808 00:37:49.139299 12425 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:49.139412 12426 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c25e8d957afb476aa914c320c4fb374b. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } }
I20260808 00:37:49.139758 12438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:49.139818 12426 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:49.140326 12438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:49.140702 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 12313
I20260808 00:37:49.140793 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:49.141023 1856 external_mini_cluster.cc:1428] Setting key 13bc1bb05d080859bfa9f77cd75c52cb
I20260808 00:37:49.142096 12438 catalog_manager.cc:1347] Generated new cluster ID: a014654d9b1144e7931c779c0de36b4a
I20260808 00:37:49.142150 12438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:49.148268 12438 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:49.148762 12438 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:49.152017 12438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Generated new TSK 0
I20260808 00:37:49.152179 12438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:49.153080 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.141681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39692 (local address 127.1.208.62:40611)
0808 00:37:49.141824 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.141834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.141866 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:37:49.141990 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.141998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.142055 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.142976 (+ 921us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.143068 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.144648 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.145780 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.146393 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.146496 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.149378 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.149382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.149382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.149395 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.151414 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.151806 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.151810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.151810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.151925 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.152192 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.152193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.152193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.152466 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.152578 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.152717 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.152838 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
I20260808 00:37:49.180871 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:49.264684 12466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:49.264897 12466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:49.264921 12466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:49.266114 12466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:49.266155 12466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:49.266182 12466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:49.267637 12466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:49.267859 12466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:49.268082 12466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.273008 12478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.273015 12481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.273052 12477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.273065 12466 server_base.cc:1034] running on GCE node
I20260808 00:37:49.273459 12466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.273685 12466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.274822 12466 hybrid_clock.cc:648] HybridClock initialized: now 1786149469274809 us; error 32 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.277279 12466 init.cc:377] Logged in from keytab as oryx/127.1.208.1@KRBTEST.COM (short username oryx)
I20260808 00:37:49.277763 12466 webserver.cc:460] Webserver started at http://127.1.208.1:35211/ using document root <none> and password file <none>
I20260808 00:37:49.277974 12466 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.278023 12466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.278149 12466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:49.279196 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "3c3650e2a50145eb9590c3bb2971f133"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "e480a206e56a12b9f2413bff7b87ac06"
server_key_iv: "af32f28a4fa5a8318927b786765e4497"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.279546 12466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "3c3650e2a50145eb9590c3bb2971f133"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "e480a206e56a12b9f2413bff7b87ac06"
server_key_iv: "af32f28a4fa5a8318927b786765e4497"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.280876 12466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:49.281669 12494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.281837 12466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.281930 12466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "3c3650e2a50145eb9590c3bb2971f133"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "e480a206e56a12b9f2413bff7b87ac06"
server_key_iv: "af32f28a4fa5a8318927b786765e4497"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.281996 12466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.302495 12466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.304447 12466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:49.304587 12466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.304819 12466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:49.305135 12466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:49.305171 12466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.305202 12466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:49.305219 12466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.310998 12466 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:43261
I20260808 00:37:49.311058 12692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:43261 every 8 connection(s)
I20260808 00:37:49.311364 12466 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:49.316629 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 12466
I20260808 00:37:49.316726 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:49.316941 1856 external_mini_cluster.cc:1428] Setting key ceaa882ccf403893d86b11d551ad862c
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:49.320775 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.312054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:55511 (local address 127.1.208.62:40611)
0808 00:37:49.312116 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.312119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.312311 (+ 192us) server_negotiation.cc:407] Connection header received
0808 00:37:49.312861 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.312862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.312875 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.312958 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.313959 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.314469 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.315410 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.315619 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.315674 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.318054 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.318055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.318056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.318063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.319439 (+ 1376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.319901 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.319902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.319902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.319959 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.320200 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.320200 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.320201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.320296 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.320340 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.320641 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.320710 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:37:49.321002 12695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.312013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:55511)
0808 00:37:49.312191 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.312229 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.312756 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.313039 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.313043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.313662 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.313859 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.313863 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.314554 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.314555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.315325 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.315328 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.315688 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.315689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.315802 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.316323 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.316335 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.317966 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.319533 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.319542 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.319545 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.319833 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.320012 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.320013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.320014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.320145 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.320348 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.320352 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.320513 (+ 161us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.320621 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.320786 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":93,"threads_started":1}
I20260808 00:37:49.321499 12693 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:49.321593 12693 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:49.321764 12693 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
I20260808 00:37:49.322377 12364 ts_manager.cc:194] Registered new tserver with Master: 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261)
I20260808 00:37:49.323282 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:55511
WARNING: no policy specified for oryx/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
I20260808 00:37:49.343626 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:49.427150 12699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:49.427356 12699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:49.427378 12699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:49.428494 12699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:49.428536 12699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:49.428561 12699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:49.430022 12699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:49.430218 12699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:49.430404 12699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.435199 12708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.435228 12713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.435228 12710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.435539 12699 server_base.cc:1034] running on GCE node
I20260808 00:37:49.435714 12699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.435943 12699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.437083 12699 hybrid_clock.cc:648] HybridClock initialized: now 1786149469437071 us; error 28 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.439541 12699 init.cc:377] Logged in from keytab as oryx/127.1.208.2@KRBTEST.COM (short username oryx)
I20260808 00:37:49.440001 12699 webserver.cc:460] Webserver started at http://127.1.208.2:44447/ using document root <none> and password file <none>
I20260808 00:37:49.440225 12699 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.440268 12699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.440402 12699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:49.441663 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "1301b35a4920489697cf5818c7237019"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "872ddac73254e6dbeffb876463ab998b"
server_key_iv: "5d366243df14e01bf86e714427a8c316"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.442039 12699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "1301b35a4920489697cf5818c7237019"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "872ddac73254e6dbeffb876463ab998b"
server_key_iv: "5d366243df14e01bf86e714427a8c316"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.443354 12699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:49.444092 12720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.444276 12699 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:49.444363 12699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "1301b35a4920489697cf5818c7237019"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "872ddac73254e6dbeffb876463ab998b"
server_key_iv: "5d366243df14e01bf86e714427a8c316"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.444445 12699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.458244 12699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.460117 12699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:49.460227 12699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.460420 12699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:49.460681 12699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:49.460711 12699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.460732 12699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:49.460748 12699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.466322 12699 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:36471
I20260808 00:37:49.466377 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:36471 every 8 connection(s)
I20260808 00:37:49.466672 12699 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:49.468954 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 12699
I20260808 00:37:49.469060 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:49.469244 1856 external_mini_cluster.cc:1428] Setting key ad07f0ed187eccf1c5d1ad4e4981b3a1
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:49.476359 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.467372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:39039 (local address 127.1.208.62:40611)
0808 00:37:49.467447 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.467449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.467613 (+ 164us) server_negotiation.cc:407] Connection header received
0808 00:37:49.468090 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.468091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.468101 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.468149 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.469131 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.469850 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.471180 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.471365 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.471419 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.473588 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.473589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.473590 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.473596 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.474899 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.475356 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.475356 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.475357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.475415 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.475684 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.475685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.475685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.475797 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.475851 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.476193 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.476268 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:37:49.476565 12906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.467342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:39039)
0808 00:37:49.467504 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.467541 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.468016 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.468204 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.468208 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.468757 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.469048 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.469055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.469951 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.469952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.471090 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.471093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.471435 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.471436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.471579 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.472098 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.472107 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.473499 (+ 1392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.474984 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.474992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.474994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.475287 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.475472 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.475473 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.475473 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.475612 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.475863 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.475869 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.476048 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.476169 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.476337 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260808 00:37:49.477094 12904 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:49.477195 12904 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:49.477423 12904 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
I20260808 00:37:49.477809 12364 ts_manager.cc:194] Registered new tserver with Master: 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:49.478188 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:39039
WARNING: no policy specified for oryx/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
I20260808 00:37:49.495604 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:49.579043 12910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:49.579254 12910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:49.579293 12910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:49.580427 12910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:49.580466 12910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:49.580500 12910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:49.582007 12910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:49.582247 12910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:49.582451 12910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.587248 12915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.587388 12910 server_base.cc:1034] running on GCE node
W20260808 00:37:49.587342 12919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.587255 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.587705 12910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.587924 12910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.589054 12910 hybrid_clock.cc:648] HybridClock initialized: now 1786149469589043 us; error 26 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.591456 12910 init.cc:377] Logged in from keytab as oryx/127.1.208.3@KRBTEST.COM (short username oryx)
I20260808 00:37:49.591867 12910 webserver.cc:460] Webserver started at http://127.1.208.3:37271/ using document root <none> and password file <none>
I20260808 00:37:49.592038 12910 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.592075 12910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.592166 12910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:49.593159 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "983133ad86d1435eb7eec17b72b76900"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "15833fdf99daa1992671f6a7f56f0670"
server_key_iv: "77634de1edf8befbd117621926dd84d4"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.593511 12910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "983133ad86d1435eb7eec17b72b76900"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "15833fdf99daa1992671f6a7f56f0670"
server_key_iv: "77634de1edf8befbd117621926dd84d4"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.594700 12910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:49.595395 12935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.595554 12910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:49.595638 12910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "983133ad86d1435eb7eec17b72b76900"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "15833fdf99daa1992671f6a7f56f0670"
server_key_iv: "77634de1edf8befbd117621926dd84d4"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.595709 12910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.604547 12910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.606278 12910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:49.606402 12910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.606599 12910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:49.606931 12910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:49.606966 12910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.606994 12910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:49.607019 12910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.612783 12910 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:38391
I20260808 00:37:49.612823 13116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:38391 every 8 connection(s)
I20260808 00:37:49.613111 12910 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:49.620795 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 12910
I20260808 00:37:49.620894 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:49.621080 1856 external_mini_cluster.cc:1428] Setting key 3fa915f5b3f08bb30c5bdc8ddf452c5a
I20260808 00:37:49.621636 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.613830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:49511 (local address 127.1.208.62:40611)
0808 00:37:49.613900 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.613903 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.614086 (+ 183us) server_negotiation.cc:407] Connection header received
0808 00:37:49.614567 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.614568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.614579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.614628 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.615524 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.616032 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.616940 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.617119 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.617160 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.619106 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.619107 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.619107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.619114 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.620312 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.620658 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.620659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.620659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.620721 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.620950 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.620951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.620952 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.621076 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.621112 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.621450 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.621535 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:37:49.621838 13121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.613800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:49511)
0808 00:37:49.613966 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.614003 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.614484 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.614684 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.614687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.615251 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.615440 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.615445 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.616096 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.616096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.616879 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.616882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.617173 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.617173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.617266 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.617703 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.617712 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.619018 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.620383 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.620389 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.620391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.620593 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.620780 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.620781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.620781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.620867 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.621156 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.621163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.621311 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.621427 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.621596 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260808 00:37:49.622277 13118 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:49.622360 13118 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:49.622510 13118 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
I20260808 00:37:49.622809 12364 ts_manager.cc:194] Registered new tserver with Master: 983133ad86d1435eb7eec17b72b76900 (127.1.208.3:38391)
I20260808 00:37:49.623162 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:49511
I20260808 00:37:49.632711 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.208.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.61.keytab.
Entry for principal oryx/127.1.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.61.keytab.
I20260808 00:37:49.658010 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:37213
--webserver_interface=127.1.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.61.keytab
--principal=oryx/127.1.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40611,127.1.208.61:37213
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:49.740171 13125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:49.740370 13125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:49.740391 13125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:49.741536 13125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:49.741577 13125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:49.741590 13125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:49.741609 13125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:49.741633 13125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:49.743098 13125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40611,127.1.208.61:37213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.61.keytab
--principal=oryx/127.1.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:37213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
W20260808 00:37:49.743208 13125 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.208.62:40611,127.1.208.61:37213'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 00:37:49.743350 13125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:49.743563 13125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.748018 13133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.748018 13132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.748068 13138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.748251 13125 server_base.cc:1034] running on GCE node
I20260808 00:37:49.748476 13125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.748706 13125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.749827 13125 hybrid_clock.cc:648] HybridClock initialized: now 1786149469749819 us; error 27 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.752187 13125 init.cc:377] Logged in from keytab as oryx/127.1.208.61@KRBTEST.COM (short username oryx)
I20260808 00:37:49.752619 13125 webserver.cc:460] Webserver started at http://127.1.208.61:37577/ using document root <none> and password file <none>
I20260808 00:37:49.752831 13125 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.752879 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.753005 13125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:49.754066 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/instance:
uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "288bec287fdf7bf8089be0e73227f6dc"
server_key_iv: "dab50b05cd85df14bf055210e0c6c8ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.754390 13125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal/instance:
uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "288bec287fdf7bf8089be0e73227f6dc"
server_key_iv: "dab50b05cd85df14bf055210e0c6c8ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.755779 13125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:49.756422 13152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.756572 13125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:49.756654 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "288bec287fdf7bf8089be0e73227f6dc"
server_key_iv: "dab50b05cd85df14bf055210e0c6c8ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.756713 13125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.774107 13125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.776173 13125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:49.776310 13125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.780321 13125 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.61:37213
I20260808 00:37:49.780375 13242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.61:37213 every 8 connection(s)
I20260808 00:37:49.780706 13125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
I20260808 00:37:49.781764 13243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:49.782692 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 13125
I20260808 00:37:49.783589 13243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.61@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
I20260808 00:37:49.792933 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.784634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42530 (local address 127.1.208.62:40611)
0808 00:37:49.784699 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.784701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.784800 (+ 99us) server_negotiation.cc:407] Connection header received
0808 00:37:49.784872 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.784873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.784881 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.784927 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.785448 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.786219 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.787431 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.787722 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.787781 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.790080 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.790082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.790082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.790091 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.791588 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.791970 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.791972 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.791972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.792035 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.792278 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.792280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.792280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.792451 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.792513 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.792759 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.792840 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:37:49.793112 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.784540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42530)
0808 00:37:49.784694 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.784739 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.784811 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.784960 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.784963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.785227 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.785367 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.785372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.786302 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.786303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.787359 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.787361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.787794 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.787795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.787887 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.788423 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.788433 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.789997 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.791659 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.791665 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.791667 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.791893 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.792112 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.792112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.792113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.792215 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.792515 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.792522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.792640 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.792728 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.792869 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
I20260808 00:37:49.793265 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.783379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50296 (local address 127.1.208.61:37213)
0808 00:37:49.783558 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.783566 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.783596 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:37:49.784146 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.784154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.784207 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.784357 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.785083 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.786412 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.787389 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.787850 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.787901 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.790544 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.790552 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.790553 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.790560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.792170 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.792473 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.792477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.792477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.792531 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.792698 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.792699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.792699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.792835 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.792966 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.793078 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.793154 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
I20260808 00:37:49.793802 13243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.61@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
I20260808 00:37:49.799132 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.794048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50298)
0808 00:37:49.794075 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.794084 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.794120 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.794214 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.794215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.794342 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.794369 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.794370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.794966 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.794967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.795428 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.795429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.795753 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.795754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.795814 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.796102 (+ 288us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.796106 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.797125 (+ 1019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.798297 (+ 1172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.798298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.798298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.798450 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.798603 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.798603 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.798604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.798655 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.798862 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.798863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.798903 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.798961 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.799038 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14}
I20260808 00:37:49.799165 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.794078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50298 (local address 127.1.208.61:37213)
0808 00:37:49.794106 (+ 28us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.794108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.794115 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:49.794167 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.794168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.794175 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.794213 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.794431 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.794901 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.795491 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.795698 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.795751 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.797194 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.797195 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.797195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.797201 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.798235 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.798506 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.798507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.798507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.798553 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.798703 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.798703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.798704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.798796 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.798864 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.798980 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.799081 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":17}
I20260808 00:37:49.800390 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Bootstrap starting.
I20260808 00:37:49.801367 13243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:49.801640 13243 log.cc:824] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:49.802307 13243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: No bootstrap required, opened a new log
I20260808 00:37:49.803673 13243 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } }
I20260808 00:37:49.803798 13243 raft_consensus.cc:374] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:49.803840 13243 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf28297c93f04a05b1b681f4ba84f5ba, State: Initialized, Role: FOLLOWER
I20260808 00:37:49.803957 13243 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } }
I20260808 00:37:49.804217 13243 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:49.804176 13251 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } } }
I20260808 00:37:49.804270 13251 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:37:49.806094 13267 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:49.806151 13267 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:49.807559 13125 master_runner.cc:428] Detected that this master cf28297c93f04a05b1b681f4ba84f5ba is joining an existing cluster
I20260808 00:37:49.807602 13125 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.208.61:37213
I20260808 00:37:49.814934 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.809230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50302)
0808 00:37:49.809448 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.809457 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.809512 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.809624 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.809625 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.809821 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.809873 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.809874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.810686 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.810687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.811231 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.811232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.811596 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.811650 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.812095 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.812098 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.812520 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.814056 (+ 1536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.814056 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.814057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.814214 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.814384 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.814385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.814385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.814438 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.814634 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.814635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.814674 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.814734 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.814829 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":20,"thread_start_us":94,"threads_started":1}
I20260808 00:37:49.815181 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.809428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50302 (local address 127.1.208.61:37213)
0808 00:37:49.809464 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.809465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.809498 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:37:49.809555 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.809555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.809564 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.809616 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.809945 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.810610 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811298 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.811489 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811542 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.812604 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.812606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.812606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.812611 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.813980 (+ 1369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.814276 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.814277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.814278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.814331 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.814487 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.814487 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.814488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.814583 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.814635 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.814759 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.814904 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":22}
I20260808 00:37:49.815937 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.808533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42546)
0808 00:37:49.809130 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.809139 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.809193 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.809366 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.809367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.809577 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.809673 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.809675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.810536 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.810537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.811455 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.811458 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.811885 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.811946 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.812228 (+ 282us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.812233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.812610 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.814791 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.814792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.814793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.815155 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.815379 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.815380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.815380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.815443 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.815695 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.815696 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.815729 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.815776 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.815855 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":1,"thread_start_us":536,"threads_started":1}
I20260808 00:37:49.816010 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.808555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.1.208.62:40611)
0808 00:37:49.808594 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.808596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.809191 (+ 595us) server_negotiation.cc:407] Connection header received
0808 00:37:49.809247 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.809248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.809258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.809326 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.809738 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.810458 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811544 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.811810 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.811879 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.812693 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.812695 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.812695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.812703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.814694 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.815224 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.815225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.815226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.815289 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.815492 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.815493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.815493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.815613 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.815670 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.815823 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.815904 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.62@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
I20260808 00:37:49.824812 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.817986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.1.208.61:37213)
0808 00:37:49.818026 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.818028 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.818209 (+ 181us) server_negotiation.cc:407] Connection header received
0808 00:37:49.818293 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.818294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.818301 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.818354 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.818844 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.819303 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.820373 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.820543 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.820596 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.822617 (+ 2021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.822618 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.822618 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.822625 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.823790 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.824130 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.824131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.824132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.824180 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.824351 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.824352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.824352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.824442 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.824494 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.824658 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.824729 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:49.824838 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.817905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50310)
0808 00:37:49.818100 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.818150 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.818224 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.818360 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.818363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.818693 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.818750 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.818752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.819374 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.819375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.820286 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.820289 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.820608 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.820609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.820687 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.821055 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.821060 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.822539 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.823864 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.823871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.823873 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.824065 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.824241 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.824242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.824242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.824302 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.824496 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.824504 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.824555 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.824627 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.824746 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":130,"threads_started":1}
I20260808 00:37:49.825352 12364 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.208.61:37213: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
member_type: NON_VOTER
last_known_addr {
host: "127.1.208.61"
port: 37213
}
attrs {
promote: true
}
}
dest_uuid: "c25e8d957afb476aa914c320c4fb374b"
cas_config_opid_index: -1
I20260808 00:37:49.825718 12364 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } }
I20260808 00:37:49.826174 13280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } }
I20260808 00:37:49.826502 13203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:49.827304 13280 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.208.61:37213
I20260808 00:37:49.827353 13203 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 00:37:49.827597 13125 master.cc:537] Master@127.1.208.61:37213 shutting down...
I20260808 00:37:49.828469 13280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:49.827625 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } }
I20260808 00:37:49.829013 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
W20260808 00:37:49.829427 12341 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 00:37:49.829874 12341 consensus_peers.cc:487] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b -> Peer cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61:37213): Couldn't send request to peer cf28297c93f04a05b1b681f4ba84f5ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:37:49.830366 13125 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:49.830446 13125 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:49.830480 13125 tablet_replica.cc:330] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: stopping tablet replica
I20260808 00:37:49.843529 13125 master.cc:552] Master@127.1.208.61:37213 shutdown complete.
I20260808 00:37:49.843583 13125 master_runner.cc:305] Clearing existing system tablet
I20260808 00:37:49.843698 13125 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.843744 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.844349 13125 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260808 00:37:49.844825 13287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.844964 13125 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.845037 13125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.845065 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "288bec287fdf7bf8089be0e73227f6dc"
server_key_iv: "dab50b05cd85df14bf055210e0c6c8ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.845989 13125 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:49.847602 13125 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:37:49.847676 13125 log.cc:1196] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 00:37:49.847872 13125 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Deleting consensus metadata
I20260808 00:37:49.847994 13125 master_runner.cc:315] Copying system tablet from 127.1.208.62:40611
I20260808 00:37:49.848618 13125 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: tablet copy: Beginning tablet copy session from remote peer at address 127.1.208.62:40611
I20260808 00:37:49.855185 13300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.848960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42562)
0808 00:37:49.849088 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:49.849099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:49.849167 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:49.849361 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:49.849362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:49.849552 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:49.849613 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.849615 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.850566 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.850567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.851145 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:49.851147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.851618 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.851619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:49.851686 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.852164 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:49.852171 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:49.852521 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:49.854328 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.854330 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.854331 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.854497 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.854685 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:49.854686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:49.854686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:49.854732 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:49.854954 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:49.854954 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:49.854990 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 00:37:49.855034 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:37:49.855107 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260808 00:37:49.855242 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.849042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.1.208.62:40611)
0808 00:37:49.849091 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.849093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.849151 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:37:49.849238 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.849239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.849249 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.849350 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.849687 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.850476 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.851233 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.851547 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.851612 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.852605 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.852607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.852607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.852613 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.854239 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.854562 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.854563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.854564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.854626 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.854779 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.854780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.854780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.854901 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.854963 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.855075 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.855152 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:37:49.855736 12395 tablet_copy_service.cc:140] P c25e8d957afb476aa914c320c4fb374b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cf28297c93f04a05b1b681f4ba84f5ba ({username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:42562)
I20260808 00:37:49.855836 12395 tablet_copy_service.cc:161] P c25e8d957afb476aa914c320c4fb374b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cf28297c93f04a05b1b681f4ba84f5ba at {username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:42562: session id = cf28297c93f04a05b1b681f4ba84f5ba-00000000000000000000000000000000
I20260808 00:37:49.856490 12395 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:37:49.857110 13125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:37:49.858415 13125 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: tablet copy: Starting download of 0 data blocks...
I20260808 00:37:49.858567 13125 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: tablet copy: Starting download of 1 WAL segments...
I20260808 00:37:49.859477 13125 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:37:49.860256 12395 tablet_copy_service.cc:342] P c25e8d957afb476aa914c320c4fb374b: Request end of tablet copy session cf28297c93f04a05b1b681f4ba84f5ba-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:42562
I20260808 00:37:49.860304 12395 tablet_copy_service.cc:433] P c25e8d957afb476aa914c320c4fb374b: ending tablet copy session cf28297c93f04a05b1b681f4ba84f5ba-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cf28297c93f04a05b1b681f4ba84f5ba
I20260808 00:37:49.862845 13125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:49.864033 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.864178 13125 server_base.cc:1034] running on GCE node
W20260808 00:37:49.864295 13305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:49.864315 13303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:49.864492 13125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:49.864643 13125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:49.865756 13125 hybrid_clock.cc:648] HybridClock initialized: now 1786149469865739 us; error 30 us; skew 500 ppm
Aug 08 00:37:49 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:49.867017 13125 init.cc:377] Logged in from keytab as oryx/127.1.208.61@KRBTEST.COM (short username oryx)
I20260808 00:37:49.867272 13125 webserver.cc:460] Webserver started at http://127.1.208.61:40491/ using document root <none> and password file <none>
I20260808 00:37:49.867386 13125 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:49.867434 13125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:49.868081 13125 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:49.868585 13320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:49.868690 13125 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:49.868731 13125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "288bec287fdf7bf8089be0e73227f6dc"
server_key_iv: "dab50b05cd85df14bf055210e0c6c8ee"
server_key_version: "encryptionkey@0"
I20260808 00:37:49.868784 13125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:49.872363 13125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:49.873796 13125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:49.878000 13125 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.61:37213
I20260808 00:37:49.878218 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.61:37213 every 8 connection(s)
I20260808 00:37:49.878343 13125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
I20260808 00:37:49.879163 13410 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:37:49.879506 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Bootstrap starting.
I20260808 00:37:49.882036 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:37:49.882256 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Bootstrap complete.
I20260808 00:37:49.882448 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } }
I20260808 00:37:49.882534 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cf28297c93f04a05b1b681f4ba84f5ba, State: Initialized, Role: LEARNER
I20260808 00:37:49.882575 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } }
I20260808 00:37:49.882751 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:49.883103 13416 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } }
I20260808 00:37:49.883210 13416 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: LEARNER
I20260808 00:37:49.884443 13432 catalog_manager.cc:1259] Loaded cluster ID: a014654d9b1144e7931c779c0de36b4a
I20260808 00:37:49.884486 13432 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: loading cluster ID for follower catalog manager: success
I20260808 00:37:49.885586 13432 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: acquiring CA information for follower catalog manager: success
I20260808 00:37:49.885897 13432 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:37:49.901965 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:49.895032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.1.208.61:37213)
0808 00:37:49.895157 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:49.895159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:49.895169 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:49.895196 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:49.895198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:49.895209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:49.895283 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:49.895768 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.896804 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.897508 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:49.897826 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:49.897903 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:49.899058 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:49.899060 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:49.899060 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:49.899068 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:49.900629 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.900948 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.900949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.900949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.901019 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:49.901161 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:49.901162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:49.901162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:49.901349 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:49.901464 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:49.901541 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 00:37:49.901624 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.1.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.60.keytab.
Entry for principal oryx/127.1.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.60.keytab.
I20260808 00:37:49.930222 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:35657
--webserver_interface=127.1.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.60.keytab
--principal=oryx/127.1.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:50.016633 13436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:50.016845 13436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:50.016867 13436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:50.018036 13436 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:50.018077 13436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:50.018090 13436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:50.018100 13436 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:50.018113 13436 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:50.019762 13436 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.60.keytab
--principal=oryx/127.1.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:50.019968 13436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:50.020143 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:50.024948 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.025024 13451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.025055 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.025250 13436 server_base.cc:1034] running on GCE node
I20260808 00:37:50.025517 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:50.025784 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:50.026922 13436 hybrid_clock.cc:648] HybridClock initialized: now 1786149470026907 us; error 29 us; skew 500 ppm
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:50.029955 13436 init.cc:377] Logged in from keytab as oryx/127.1.208.60@KRBTEST.COM (short username oryx)
I20260808 00:37:50.030406 13436 webserver.cc:460] Webserver started at http://127.1.208.60:37799/ using document root <none> and password file <none>
I20260808 00:37:50.030606 13436 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.030652 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.030778 13436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:50.031948 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/instance:
uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
format_stamp: "Formatted at 2026-08-08 00:37:50 on dist-test-slave-gjw7"
server_key: "5bdc844ce7747d25808c5bd924864219"
server_key_iv: "0fc03fb1d57748a789c4dd5ab93dd408"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.032332 13436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal/instance:
uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
format_stamp: "Formatted at 2026-08-08 00:37:50 on dist-test-slave-gjw7"
server_key: "5bdc844ce7747d25808c5bd924864219"
server_key_iv: "0fc03fb1d57748a789c4dd5ab93dd408"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.033941 13436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:50.034672 13461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.034829 13436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:50.034925 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
format_stamp: "Formatted at 2026-08-08 00:37:50 on dist-test-slave-gjw7"
server_key: "5bdc844ce7747d25808c5bd924864219"
server_key_iv: "0fc03fb1d57748a789c4dd5ab93dd408"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.034986 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.053638 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:50.055558 13436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:50.055740 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:50.068533 13436 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.60:35657
I20260808 00:37:50.068619 13551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.60:35657 every 8 connection(s)
I20260808 00:37:50.069026 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
I20260808 00:37:50.070361 13557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:50.072933 13557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:50.076435 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 13436
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.60@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
I20260808 00:37:50.083688 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.073781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42574 (local address 127.1.208.62:40611)
0808 00:37:50.073859 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.073861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.074012 (+ 151us) server_negotiation.cc:407] Connection header received
0808 00:37:50.074818 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.074820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.074834 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.074886 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.075747 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.076276 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.077753 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.078063 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.078135 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.080891 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.080892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.080893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.080900 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.082233 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.082688 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.082689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.082689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.082750 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.083012 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.083012 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.083012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.083151 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.083214 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.083540 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.083618 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:37:50.084117 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.073628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42574)
0808 00:37:50.073896 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.073935 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.074708 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.074957 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.074960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.075276 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.075653 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.075659 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.076365 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.076366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.077670 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.077674 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.078134 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.078135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.078322 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.078948 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.078958 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.080805 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.082305 (+ 1500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.082312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.082314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.082598 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.082809 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.082809 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.082810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.082949 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.083208 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.083215 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.083374 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.083509 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.083842 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":171,"threads_started":1}
I20260808 00:37:50.084892 13557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:37:50.086755 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.076977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.1.208.60:35657)
0808 00:37:50.077339 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.077345 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.077365 (+ 20us) server_negotiation.cc:407] Connection header received
0808 00:37:50.077385 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.077387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.077427 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.077496 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.078181 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.079595 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.080380 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.080799 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.080863 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.083456 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.083464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.083464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.083471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.085473 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.085817 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.085820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.085820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.085879 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.086050 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.086051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.086051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.086184 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.086465 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.086514 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.086640 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":272,"thread_start_us":205,"threads_started":1}
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.60@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
I20260808 00:37:50.091253 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.085164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50342)
0808 00:37:50.085268 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.085278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.085365 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.085539 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.085540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.085681 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.085820 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.085822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.086450 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.086451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.087137 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.087140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.087540 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.087541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.087677 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.087957 (+ 280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.087960 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.089033 (+ 1073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.090373 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.090374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.090374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.090533 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.090703 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.090703 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.090704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.090759 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.090975 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.090976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.091020 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.091078 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.091166 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":24}
I20260808 00:37:50.091279 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.085225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50342 (local address 127.1.208.61:37213)
0808 00:37:50.085265 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.085267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.085340 (+ 73us) server_negotiation.cc:407] Connection header received
0808 00:37:50.085432 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.085433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.085448 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.085493 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.085890 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.086375 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.087217 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.087477 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.087534 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.089112 (+ 1578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.089113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.089114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.089122 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.090300 (+ 1178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.090594 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.090595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.090596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.090650 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.090808 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.090809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.090809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.090921 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.090975 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.091109 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.091191 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:37:50.091889 13557 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.60@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
I20260808 00:37:50.097481 13561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.092176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.0.0.1:47744)
0808 00:37:50.092224 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.092234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.092282 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.092384 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.092385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.092493 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.092520 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.092522 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.093224 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.093225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.093724 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.093725 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.094109 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.094110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.094142 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.094418 (+ 276us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.094422 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.095443 (+ 1021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.096778 (+ 1335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.096779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.096779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.096914 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.097033 (+ 119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.097034 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.097034 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.097076 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.097225 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.097225 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.097255 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.097342 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.097423 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:37:50.097556 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.092208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47744 (local address 127.1.208.60:35657)
0808 00:37:50.092243 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.092244 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.092267 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:37:50.092331 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.092332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.092339 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.092375 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.092564 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.093168 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.093804 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.094055 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.094107 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.095525 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.095526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.095527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.095533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.096722 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.096955 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.096956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.096956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.096994 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.097110 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.097111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.097111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.097189 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.097227 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.097366 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.097466 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":16}
I20260808 00:37:50.099100 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Bootstrap starting.
I20260808 00:37:50.100730 13557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:50.101620 13557 log.cc:824] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:50.102453 13557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: No bootstrap required, opened a new log
I20260808 00:37:50.105952 13557 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } }
I20260808 00:37:50.106082 13557 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:50.106114 13557 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcaa2f6aab1e4bcaa3c56f941e397716, State: Initialized, Role: FOLLOWER
I20260808 00:37:50.106237 13557 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } }
I20260808 00:37:50.106637 13557 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:50.106774 13569 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } } }
I20260808 00:37:50.106916 13569 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:37:50.109683 13588 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:37:50.109730 13588 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:37:50.111689 13436 master_runner.cc:428] Detected that this master fcaa2f6aab1e4bcaa3c56f941e397716 is joining an existing cluster
I20260808 00:37:50.111729 13436 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.208.60:35657
I20260808 00:37:50.121680 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.0.0.1:47760)
0808 00:37:50.113729 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.113740 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.113794 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.114045 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.114046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.114497 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.114852 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.114854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.115683 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.115683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117196 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.117200 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117763 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117875 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.118439 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.118443 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.118949 (+ 506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.120646 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.120647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.120647 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.120804 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.121001 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.121001 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.121001 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.121048 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.121319 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.121320 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.121362 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.121495 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.121578 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":369,"threads_started":1}
I20260808 00:37:50.121757 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47760 (local address 127.1.208.60:35657)
0808 00:37:50.113944 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.113946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.113954 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:50.113973 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.113973 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.113982 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.114039 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.114933 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.115609 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117283 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117582 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117617 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.119091 (+ 1474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.119093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.119093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.119102 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.120573 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.120871 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.120872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.120873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.120938 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.121098 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.121099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.121099 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.121237 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.121345 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.121538 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.121643 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":30}
I20260808 00:37:50.137327 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.1.208.61:37213)
0808 00:37:50.113266 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.113268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.114605 (+ 1337us) server_negotiation.cc:407] Connection header received
0808 00:37:50.114708 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.114709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.114719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.114751 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.115483 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.116209 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117354 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117639 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117674 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.134389 (+ 16715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.134391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.134391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.134400 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.135997 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136512 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.136513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.136513 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.136566 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136779 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.136780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.136780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.136894 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.136945 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.137114 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.137204 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:50.137370 13602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50348)
0808 00:37:50.114533 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.114544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.114633 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.114817 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.114818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.115221 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.115387 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.115389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.116306 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.116307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117251 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.117253 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117801 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117923 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.133754 (+ 15831us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.133760 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.134275 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.136100 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136101 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.136102 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.136425 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.136620 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136620 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.136621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.136706 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.136947 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.136948 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.136996 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.137058 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.137168 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":727,"mutex_wait_us":40,"thread_start_us":323,"threads_started":1}
I20260808 00:37:50.137866 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42580 (local address 127.1.208.62:40611)
0808 00:37:50.113398 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.113400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.113851 (+ 451us) server_negotiation.cc:407] Connection header received
0808 00:37:50.113939 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.113940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.113951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.113996 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.114983 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.115800 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117372 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117657 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117723 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.134630 (+ 16907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.134633 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.134633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.134641 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.135968 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136376 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.136378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.136378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.136439 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136975 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.136976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.136976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.137096 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.137150 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.137697 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.137778 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:50.138051 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.113235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42580)
0808 00:37:50.113782 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.113791 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.113874 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.113998 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.113999 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.114493 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.114873 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.114876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.115905 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.115907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117234 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.117237 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.117781 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.117782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.117888 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.134076 (+ 16188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.134081 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.134544 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.136050 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.136052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.136304 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.136622 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.136623 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.136623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.136713 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.137179 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.137179 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.137222 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.137274 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.137396 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":37,"thread_start_us":341,"threads_started":1}
I20260808 00:37:50.138125 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 12466
I20260808 00:37:50.170265 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:43261
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:40611
--webserver_port=35211
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149469, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.62@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
I20260808 00:37:50.173450 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.151228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.0.0.1:47770)
0808 00:37:50.165427 (+ 14199us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.165444 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.165531 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.165829 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.165831 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.166050 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.166112 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.166114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.167039 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.167040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.167841 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.167845 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.168408 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.168409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.168483 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.169023 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.169029 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.170649 (+ 1620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.172321 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.172322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.172323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.172542 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.172773 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.172773 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.172774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.172834 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.173101 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.173102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.173148 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.173205 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.173339 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14167}
I20260808 00:37:50.173779 13562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.151307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.1.208.60:35657)
0808 00:37:50.151344 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.151346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.165529 (+ 14183us) server_negotiation.cc:407] Connection header received
0808 00:37:50.165711 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.165712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.165724 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.165768 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.166197 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.166941 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.167939 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.168238 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.168342 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.170749 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.170751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.170751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.170761 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.172236 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.172633 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.172635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.172635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.172696 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.172894 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.172895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.172895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.173023 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.173076 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.173591 (+ 515us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.173679 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:37:50.174280 12364 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.208.60:35657: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
member_type: NON_VOTER
last_known_addr {
host: "127.1.208.60"
port: 35657
}
attrs {
promote: true
}
}
dest_uuid: "c25e8d957afb476aa914c320c4fb374b"
cas_config_opid_index: 5
I20260808 00:37:50.174551 12364 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } }
I20260808 00:37:50.174904 13280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.175246 13513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:50.175843 13280 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.208.60:35657
I20260808 00:37:50.175917 13280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.176000 13280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.176270 13513 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:37:50.176885 13436 master.cc:537] Master@127.1.208.60:35657 shutting down...
I20260808 00:37:50.178153 13279 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 00:37:50.179260 12346 consensus_peers.cc:487] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b -> Peer fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657): Couldn't send request to peer fcaa2f6aab1e4bcaa3c56f941e397716. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:37:50.180294 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.176165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50356)
0808 00:37:50.176333 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.176343 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.176374 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.176906 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.176907 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.176909 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.176950 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.176951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.177926 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.177927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.179139 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.179142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.180039 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.180041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.180106 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.180132 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.180178 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.180217 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260808 00:37:50.180541 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.176474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.1.208.61:37213)
0808 00:37:50.176540 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.176542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.176551 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:50.176578 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.176579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.176606 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.176682 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.177031 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.177830 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.179237 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.179852 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.179921 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.180003 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.180432 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.180474 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:37:50.180868 13369 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 LEARNER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 00:37:50.181151 13279 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 00:37:50.181878 13369 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.182982 13609 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.183089 13609 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: LEARNER
I20260808 00:37:50.183328 13436 raft_consensus.cc:2229] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:50.183411 13436 raft_consensus.cc:2258] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:50.183454 13436 tablet_replica.cc:330] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: stopping tablet replica
I20260808 00:37:50.183478 13416 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader c25e8d957afb476aa914c320c4fb374b. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: NON_VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: true } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.183560 13416 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: LEARNER
I20260808 00:37:50.183915 13279 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: attempting to promote NON_VOTER cf28297c93f04a05b1b681f4ba84f5ba to VOTER
I20260808 00:37:50.184098 13279 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } }
W20260808 00:37:50.184835 12346 consensus_peers.cc:487] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b -> Peer fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657): Couldn't send request to peer fcaa2f6aab1e4bcaa3c56f941e397716. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 00:37:50.185199 13369 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 LEARNER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 00:37:50.185397 13280 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 00:37:50.186518 13610 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.186674 13369 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.187567 13416 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.187649 13416 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:50.187594 13610 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.187808 13610 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.187718 13279 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.187906 13279 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.197268 13436 master.cc:552] Master@127.1.208.60:35657 shutdown complete.
I20260808 00:37:50.197376 13436 master_runner.cc:305] Clearing existing system tablet
I20260808 00:37:50.197507 13436 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.197553 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.198289 13436 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:50.198848 13612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.199005 13436 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.199066 13436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:50.199108 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
format_stamp: "Formatted at 2026-08-08 00:37:50 on dist-test-slave-gjw7"
server_key: "5bdc844ce7747d25808c5bd924864219"
server_key_iv: "0fc03fb1d57748a789c4dd5ab93dd408"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.199946 13436 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:50.201395 13436 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 00:37:50.201465 13436 log.cc:1196] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 00:37:50.201651 13436 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Deleting consensus metadata
I20260808 00:37:50.201763 13436 master_runner.cc:315] Copying system tablet from 127.1.208.62:40611
I20260808 00:37:50.202373 13436 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: tablet copy: Beginning tablet copy session from remote peer at address 127.1.208.62:40611
I20260808 00:37:50.209766 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.202765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42594 (local address 127.1.208.62:40611)
0808 00:37:50.202820 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.202822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.202871 (+ 49us) server_negotiation.cc:407] Connection header received
0808 00:37:50.202939 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.202940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.202951 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.203029 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.203398 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.204174 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.205171 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.205525 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.205589 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.206696 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.206698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.206698 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.206707 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.208396 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.208847 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.208849 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.208850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.208922 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.209134 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.209135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.209136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.209276 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.209367 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.209574 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.209657 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:37:50.210419 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.202680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:42594)
0808 00:37:50.202810 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.202820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.202885 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.203041 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.203042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.203249 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.203310 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.203312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.204262 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.204263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.205082 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.205085 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.205615 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.205617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.205688 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.206169 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.206175 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.206583 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.208480 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.208481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.208481 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.208725 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.208988 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.208989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.208989 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.209070 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.209399 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.209401 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.209454 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.210201 (+ 747us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.210320 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":73,"threads_started":1}
I20260808 00:37:50.210899 12395 tablet_copy_service.cc:140] P c25e8d957afb476aa914c320c4fb374b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer fcaa2f6aab1e4bcaa3c56f941e397716 ({username='oryx', principal='oryx/127.1.208.60@KRBTEST.COM'} at 127.0.0.1:42594)
I20260808 00:37:50.210964 12395 tablet_copy_service.cc:161] P c25e8d957afb476aa914c320c4fb374b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer fcaa2f6aab1e4bcaa3c56f941e397716 at {username='oryx', principal='oryx/127.1.208.60@KRBTEST.COM'} at 127.0.0.1:42594: session id = fcaa2f6aab1e4bcaa3c56f941e397716-00000000000000000000000000000000
I20260808 00:37:50.211565 12395 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 00:37:50.212163 13436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 00:37:50.213581 13436 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: tablet copy: Starting download of 0 data blocks...
I20260808 00:37:50.213734 13436 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: tablet copy: Starting download of 1 WAL segments...
I20260808 00:37:50.214497 13436 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 00:37:50.215415 12395 tablet_copy_service.cc:342] P c25e8d957afb476aa914c320c4fb374b: Request end of tablet copy session fcaa2f6aab1e4bcaa3c56f941e397716-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.208.60@KRBTEST.COM'} at 127.0.0.1:42594
I20260808 00:37:50.215464 12395 tablet_copy_service.cc:433] P c25e8d957afb476aa914c320c4fb374b: ending tablet copy session fcaa2f6aab1e4bcaa3c56f941e397716-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer fcaa2f6aab1e4bcaa3c56f941e397716
I20260808 00:37:50.218811 13436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:50.219964 13624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.220233 13628 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.220431 13436 server_base.cc:1034] running on GCE node
W20260808 00:37:50.220239 13625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.220664 13436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:50.220826 13436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:50.221956 13436 hybrid_clock.cc:648] HybridClock initialized: now 1786149470221947 us; error 24 us; skew 500 ppm
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:50.223482 13436 init.cc:377] Logged in from keytab as oryx/127.1.208.60@KRBTEST.COM (short username oryx)
I20260808 00:37:50.223771 13436 webserver.cc:460] Webserver started at http://127.1.208.60:34007/ using document root <none> and password file <none>
I20260808 00:37:50.223877 13436 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.223904 13436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.224593 13436 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:50.225139 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.225324 13436 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.225378 13436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
format_stamp: "Formatted at 2026-08-08 00:37:50 on dist-test-slave-gjw7"
server_key: "5bdc844ce7747d25808c5bd924864219"
server_key_iv: "0fc03fb1d57748a789c4dd5ab93dd408"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.225430 13436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.235893 13436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:50.237780 13436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:50.241097 13436 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.60:35657
I20260808 00:37:50.241503 13724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.60:35657 every 8 connection(s)
I20260808 00:37:50.241534 13436 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
I20260808 00:37:50.242443 13725 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:37:50.242961 13725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Bootstrap starting.
I20260808 00:37:50.245858 13725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:37:50.246075 13725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: Bootstrap complete.
I20260808 00:37:50.246274 13725 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } }
I20260808 00:37:50.246366 13725 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: fcaa2f6aab1e4bcaa3c56f941e397716, State: Initialized, Role: LEARNER
I20260808 00:37:50.246403 13725 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } }
I20260808 00:37:50.246599 13725 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:50.246619 13728 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.246846 13728 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: LEARNER
I20260808 00:37:50.248031 13747 catalog_manager.cc:1259] Loaded cluster ID: a014654d9b1144e7931c779c0de36b4a
I20260808 00:37:50.248106 13747 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: loading cluster ID for follower catalog manager: success
I20260808 00:37:50.249249 13747 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: acquiring CA information for follower catalog manager: success
I20260808 00:37:50.249599 13747 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 00:37:50.268359 13604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:50.268555 13604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:50.268577 13604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:50.269800 13604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:50.269841 13604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:50.269866 13604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:50.271330 13604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:43261
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=35211
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:50.271549 13604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:50.271767 13604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:50.277627 13757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.277644 13759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.277612 13755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.277916 13604 server_base.cc:1034] running on GCE node
I20260808 00:37:50.278100 13604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:50.278321 13604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:50.279472 13604 hybrid_clock.cc:648] HybridClock initialized: now 1786149470279445 us; error 45 us; skew 500 ppm
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:50.282346 13604 init.cc:377] Logged in from keytab as oryx/127.1.208.1@KRBTEST.COM (short username oryx)
I20260808 00:37:50.282895 13604 webserver.cc:460] Webserver started at http://127.1.208.1:35211/ using document root <none> and password file <none>
I20260808 00:37:50.283128 13604 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.283197 13604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.285157 13604 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:37:50.285864 13773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.286069 13604 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:50.286165 13604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "3c3650e2a50145eb9590c3bb2971f133"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "e480a206e56a12b9f2413bff7b87ac06"
server_key_iv: "af32f28a4fa5a8318927b786765e4497"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.286616 13604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.301393 13604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:50.303381 13604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:50.303553 13604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:50.303965 13604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:50.304311 13604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:50.304345 13604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.304368 13604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:50.304383 13604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.321870 13604 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:43261
I20260808 00:37:50.321916 13961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:43261 every 8 connection(s)
I20260808 00:37:50.322329 13604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:50.327068 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 13604
I20260808 00:37:50.327257 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 12699
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:50.337579 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:39401 (local address 127.1.208.62:40611)
0808 00:37:50.323593 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.323595 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.323874 (+ 279us) server_negotiation.cc:407] Connection header received
0808 00:37:50.325152 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.325154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.325168 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.325214 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.326299 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.327067 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.331133 (+ 4066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.331437 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.331483 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.334180 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.334181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.334182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.334190 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.335663 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.336095 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.336096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.336096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.336152 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.337055 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.337056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.337057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.337189 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.337231 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.337419 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.337494 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:37:50.338306 13971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:39401)
0808 00:37:50.323816 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.323825 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.325026 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.325333 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.325337 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.325867 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.326181 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.326188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.327235 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.327236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.331029 (+ 3793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.331032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.331574 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.331575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.331699 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.332136 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.332141 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.334068 (+ 1927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.335740 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.335741 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.335741 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.336015 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.336216 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.336216 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.336217 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.336446 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.337260 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.337261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.337317 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.337854 (+ 537us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.338026 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":3,"thread_start_us":507,"threads_started":1}
I20260808 00:37:50.338843 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:41677 (local address 127.1.208.60:35657)
0808 00:37:50.323381 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.323384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.324022 (+ 638us) server_negotiation.cc:407] Connection header received
0808 00:37:50.325364 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.325365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.325380 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.325491 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.326298 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.327319 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.328842 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.329598 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.329647 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.333120 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.333121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.333122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.333130 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.335512 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.336085 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.336086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.336087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.336152 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.336578 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.336579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.336579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.336702 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.336737 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.338650 (+ 1913us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.338748 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260808 00:37:50.339385 13972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.1.208.1:41677)
0808 00:37:50.323918 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.323928 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.325049 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.325568 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.325569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.325930 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.326208 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.326210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.327399 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.327400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.328730 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.328733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.329740 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.329741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.329892 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.330568 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.330579 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.333000 (+ 2421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.335610 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.335619 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.335622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.335992 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.336220 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.336220 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.336221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.336520 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.336770 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.336776 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.336927 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.339137 (+ 2210us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.339231 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":42,"thread_start_us":379,"threads_started":1}
I20260808 00:37:50.340798 13963 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:50.340905 13963 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.341141 13963 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
I20260808 00:37:50.341683 12364 ts_manager.cc:194] Re-registered known tserver with Master: 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261)
I20260808 00:37:50.342165 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:39401
I20260808 00:37:50.342417 13962 heartbeater.cc:347] Connected to a master server at 127.1.208.60:35657
I20260808 00:37:50.342465 13962 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.342525 13962 heartbeater.cc:511] Master 127.1.208.60:35657 requested a full tablet report, sending...
I20260808 00:37:50.342958 13663 ts_manager.cc:194] Registered new tserver with Master: 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261)
I20260808 00:37:50.344475 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34833 (local address 127.1.208.61:37213)
0808 00:37:50.323552 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.323553 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.323786 (+ 233us) server_negotiation.cc:407] Connection header received
0808 00:37:50.325221 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.325223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.325238 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.325310 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.326294 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.326825 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.328453 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.328677 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.328718 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.341849 (+ 13131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.341851 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.341851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.341861 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.343447 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.343752 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.343753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.343754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.343812 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.343984 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.343985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.343985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.344104 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.344159 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.344293 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.344376 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:50.344543 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.323138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.1.208.1:34833)
0808 00:37:50.323655 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.323699 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.325026 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.325335 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.325339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.325802 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.326181 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.326187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.327069 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.327070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.328364 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.328367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.328793 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.328793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.328965 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.340232 (+ 11267us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.340236 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.341762 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.343516 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.343517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.343517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.343683 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.343868 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.343869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.343869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.343932 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.344162 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.344163 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.344206 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.344265 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.344376 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":440,"threads_started":1}
I20260808 00:37:50.344918 13966 heartbeater.cc:347] Connected to a master server at 127.1.208.61:37213
I20260808 00:37:50.344964 13966 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.345023 13966 heartbeater.cc:511] Master 127.1.208.61:37213 requested a full tablet report, sending...
I20260808 00:37:50.345499 13344 ts_manager.cc:194] Registered new tserver with Master: 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261)
I20260808 00:37:50.345994 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:36471
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:40611
--webserver_port=44447
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:50.431291 13974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:50.431495 13974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:50.431520 13974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:50.432657 13974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:50.432698 13974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:50.432726 13974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:50.434206 13974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:36471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=44447
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:50.434441 13974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:50.434656 13974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:50.440160 13985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.440176 13981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.440243 13982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.440501 13974 server_base.cc:1034] running on GCE node
I20260808 00:37:50.440690 13974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:50.440909 13974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:50.442045 13974 hybrid_clock.cc:648] HybridClock initialized: now 1786149470442027 us; error 27 us; skew 500 ppm
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:50.444499 13974 init.cc:377] Logged in from keytab as oryx/127.1.208.2@KRBTEST.COM (short username oryx)
I20260808 00:37:50.444875 13974 webserver.cc:460] Webserver started at http://127.1.208.2:44447/ using document root <none> and password file <none>
I20260808 00:37:50.445047 13974 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.445101 13974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.446449 13974 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:50.447113 13995 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.447265 13974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:50.447366 13974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "1301b35a4920489697cf5818c7237019"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "872ddac73254e6dbeffb876463ab998b"
server_key_iv: "5d366243df14e01bf86e714427a8c316"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.447690 13974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.457729 13974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:50.459367 13974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:50.459474 13974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:50.459646 13974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:50.459949 13974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:50.459985 13974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.460005 13974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:50.460019 13974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.466903 13974 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:36471
I20260808 00:37:50.466952 14175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:36471 every 8 connection(s)
I20260808 00:37:50.467272 13974 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:37:50.471650 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 13974
I20260808 00:37:50.471812 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 12910
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
I20260808 00:37:50.486811 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.468141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:40815 (local address 127.1.208.60:35657)
0808 00:37:50.468191 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.468193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.468529 (+ 336us) server_negotiation.cc:407] Connection header received
0808 00:37:50.469074 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.469075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.469092 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.469143 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.471265 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.471884 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.479357 (+ 7473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.479698 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.479742 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.483277 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.483279 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.483280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.483291 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.485107 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.485654 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.485655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.485656 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.485739 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.486008 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.486009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.486009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.486188 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.486269 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.486583 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.486699 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:50.486958 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.467992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.1.208.2:40815)
0808 00:37:50.468400 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.468418 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.468982 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.469623 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.469624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.469859 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.470044 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.470046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.478167 (+ 8121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.478169 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.479201 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.479206 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.479842 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.479843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.479998 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.481203 (+ 1205us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.481215 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.483168 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.485200 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.485210 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.485213 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.485553 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.485807 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.485808 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.485808 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.485933 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.486253 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.486258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.486427 (+ 169us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.486514 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.486672 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":4,"thread_start_us":273,"threads_started":1}
I20260808 00:37:50.487674 14176 heartbeater.cc:347] Connected to a master server at 127.1.208.60:35657
I20260808 00:37:50.487829 14176 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.488126 14176 heartbeater.cc:511] Master 127.1.208.60:35657 requested a full tablet report, sending...
I20260808 00:37:50.488703 13663 ts_manager.cc:194] Registered new tserver with Master: 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:50.499917 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.468161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:57565 (local address 127.1.208.62:40611)
0808 00:37:50.468207 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.468209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.468614 (+ 405us) server_negotiation.cc:407] Connection header received
0808 00:37:50.469101 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.469102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.469116 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.469147 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.470706 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.471184 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.494225 (+ 23041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.494762 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.494820 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.497587 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.497589 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.497589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.497598 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.498925 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499217 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.499218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.499218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.499271 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499436 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.499436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.499437 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.499547 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.499599 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.499744 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.499817 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:50.499938 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.468009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:57565)
0808 00:37:50.468543 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.468551 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.468989 (+ 438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.469234 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.469235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.469731 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.470027 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.470034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.471262 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.471263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.494109 (+ 22846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.494114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.494880 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.494881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.495011 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.495534 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.495538 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.497501 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.498990 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.498991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.498991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.499148 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.499321 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.499322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.499379 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.499599 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.499600 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.499647 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.499701 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.499811 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":22,"spinlock_wait_cycles":7936,"thread_start_us":467,"threads_started":1}
I20260808 00:37:50.499976 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.468152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:60147 (local address 127.1.208.61:37213)
0808 00:37:50.468219 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.468222 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.468499 (+ 277us) server_negotiation.cc:407] Connection header received
0808 00:37:50.469101 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.469103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.469116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.469147 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.470124 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.470623 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.471831 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.472082 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.472118 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.497141 (+ 25023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.497144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.497144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.497154 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.498716 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499117 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.499119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.499119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.499185 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499391 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.499392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.499392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.499546 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.499592 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.499788 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.499879 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:37:50.500548 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:38391
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:40611
--webserver_port=37271
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:37:50.500599 14178 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:50.500617 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.467993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.1.208.2:60147)
0808 00:37:50.468371 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.468415 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.468976 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.469228 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.469233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.469714 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.470027 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.470034 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.470706 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.470707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.471734 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.471736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.494259 (+ 22523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.494259 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.494335 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.495405 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.495410 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.497029 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.498805 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.498806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.498806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.499022 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.499258 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.499259 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.499259 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.499317 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.499640 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.499641 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.499684 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.500410 (+ 726us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.500515 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":9,"thread_start_us":183,"threads_started":1}
I20260808 00:37:50.500669 14178 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.500747 14178 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
I20260808 00:37:50.500972 12364 ts_manager.cc:194] Re-registered known tserver with Master: 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:50.501389 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:57565
I20260808 00:37:50.502612 14179 heartbeater.cc:347] Connected to a master server at 127.1.208.61:37213
I20260808 00:37:50.502663 14179 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.502751 14179 heartbeater.cc:511] Master 127.1.208.61:37213 requested a full tablet report, sending...
I20260808 00:37:50.503054 13344 ts_manager.cc:194] Registered new tserver with Master: 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
W20260808 00:37:50.584807 14185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:50.585016 14185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:50.585039 14185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:50.586177 14185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:50.586218 14185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:50.586242 14185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:50.587666 14185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:38391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=37271
--tserver_master_addrs=127.1.208.62:40611,127.1.208.61:37213,127.1.208.60:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:50.587859 14185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:50.588044 14185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:50.592993 14197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:50.593019 14195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.593120 14185 server_base.cc:1034] running on GCE node
W20260808 00:37:50.593129 14191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:50.593425 14185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:50.593626 14185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:50.594760 14185 hybrid_clock.cc:648] HybridClock initialized: now 1786149470594752 us; error 27 us; skew 500 ppm
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:50.597074 14185 init.cc:377] Logged in from keytab as oryx/127.1.208.3@KRBTEST.COM (short username oryx)
I20260808 00:37:50.597519 14185 webserver.cc:460] Webserver started at http://127.1.208.3:37271/ using document root <none> and password file <none>
I20260808 00:37:50.597718 14185 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:50.597782 14185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:50.599045 14185 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:50.599529 14213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.599673 14185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.599753 14185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "983133ad86d1435eb7eec17b72b76900"
format_stamp: "Formatted at 2026-08-08 00:37:49 on dist-test-slave-gjw7"
server_key: "15833fdf99daa1992671f6a7f56f0670"
server_key_iv: "77634de1edf8befbd117621926dd84d4"
server_key_version: "encryptionkey@0"
I20260808 00:37:50.600082 14185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:50.612291 14185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:50.614009 14185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:50.614135 14185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:50.614334 14185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:37:50.614627 14185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:50.614666 14185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.614694 14185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:50.614718 14185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:50.620554 14185 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:38391
I20260808 00:37:50.620596 14385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:38391 every 8 connection(s)
I20260808 00:37:50.620935 14185 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:50.626176 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 14185
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.61@KRBTEST.COM
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.60@KRBTEST.COM
Aug 08 00:37:50 dist-test-slave-gjw7 krb5kdc[12292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149470, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:50.632913 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:53055 (local address 127.1.208.61:37213)
0808 00:37:50.621840 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.621842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.622196 (+ 354us) server_negotiation.cc:407] Connection header received
0808 00:37:50.623351 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.623352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.623365 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.623430 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.624253 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.625045 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626412 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626702 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626744 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.629720 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.629722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.629722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.629729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.631343 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631853 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.631854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.631854 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.631915 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632205 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.632207 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.632207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.632336 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.632378 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.632732 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.632809 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:50.633131 14404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.1.208.3:50537)
0808 00:37:50.622383 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.622392 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.623173 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.623529 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.623530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.623878 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.624171 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.624173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.625342 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.625343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.626313 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.626316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626860 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.627003 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.627628 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.627632 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.629538 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.631281 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631289 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.631291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.631754 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.631975 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631976 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.631976 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.632128 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.632424 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.632431 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.632583 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.632688 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.632845 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":142,"thread_start_us":149,"threads_started":1}
I20260808 00:37:50.633548 12463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:54169 (local address 127.1.208.62:40611)
0808 00:37:50.621840 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.621842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.622236 (+ 394us) server_negotiation.cc:407] Connection header received
0808 00:37:50.623271 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.623273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.623287 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.623345 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.624238 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.624767 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626456 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626744 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626779 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.630376 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.630377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.630378 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.630385 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.632249 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632602 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.632603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.632604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.632660 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632849 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.632849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.632850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.632969 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.633009 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.633377 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.633454 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:37:50.633610 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:50537 (local address 127.1.208.60:35657)
0808 00:37:50.621807 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.621809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.622436 (+ 627us) server_negotiation.cc:407] Connection header received
0808 00:37:50.623321 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.623322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.623342 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.623371 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.624257 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.625040 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626435 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626735 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626776 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.629637 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.629639 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.629639 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.629646 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.631203 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631838 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.631839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.631840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.631904 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632199 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.632200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.632200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.632334 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.632387 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.633435 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.633518 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:50.633888 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:54169)
0808 00:37:50.622071 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.622097 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.623172 (+ 1075us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.623416 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.623420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.623898 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.624147 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.624150 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.625004 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.625005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.626370 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.626373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626863 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.627015 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.628229 (+ 1214us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.628233 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.630260 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.632340 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.632341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.632523 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.632728 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.632729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.632729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.632781 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.633101 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.633101 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.633226 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.633667 (+ 441us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.633759 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":91,"thread_start_us":307,"threads_started":1}
I20260808 00:37:50.633996 14386 heartbeater.cc:347] Connected to a master server at 127.1.208.60:35657
I20260808 00:37:50.634094 14386 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.634150 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.621657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.1.208.3:53055)
0808 00:37:50.622054 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.622097 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.623166 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.623438 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.623439 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.623847 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:50.624137 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.624144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.625115 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.625116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.626312 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.626315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.626856 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.626857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.627001 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.627482 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:50.627490 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:50.629564 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:50.631431 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.631432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.631754 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.631986 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:50.631987 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:50.631987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:50.632129 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:50.632421 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:50.632428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:50.632581 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.633397 (+ 816us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.633486 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"mutex_wait_us":90,"thread_start_us":259,"threads_started":1}
I20260808 00:37:50.634267 14393 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40611
I20260808 00:37:50.634388 14393 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.634460 14393 heartbeater.cc:511] Master 127.1.208.62:40611 requested a full tablet report, sending...
I20260808 00:37:50.634845 12364 ts_manager.cc:194] Re-registered known tserver with Master: 983133ad86d1435eb7eec17b72b76900 (127.1.208.3:38391)
I20260808 00:37:50.635186 12364 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:54169
I20260808 00:37:50.635372 14399 heartbeater.cc:347] Connected to a master server at 127.1.208.61:37213
I20260808 00:37:50.635460 14399 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:50.635556 14399 heartbeater.cc:511] Master 127.1.208.61:37213 requested a full tablet report, sending...
I20260808 00:37:50.636282 13344 ts_manager.cc:194] Registered new tserver with Master: 983133ad86d1435eb7eec17b72b76900 (127.1.208.3:38391)
I20260808 00:37:50.634280 14386 heartbeater.cc:511] Master 127.1.208.60:35657 requested a full tablet report, sending...
I20260808 00:37:50.637028 13663 ts_manager.cc:194] Registered new tserver with Master: 983133ad86d1435eb7eec17b72b76900 (127.1.208.3:38391)
I20260808 00:37:50.640522 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.628183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.1.208.61:37213)
0808 00:37:50.628488 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.628490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.631395 (+ 2905us) server_negotiation.cc:407] Connection header received
0808 00:37:50.631487 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.631489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.631501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.631539 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.634032 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.634707 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.635707 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.636011 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.636075 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.637774 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.637775 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.637776 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.637783 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.639451 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.639778 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.639779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.639779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.639856 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.640014 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.640015 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.640015 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.640115 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.640203 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.640348 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.640430 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":65,"threads_started":1}
I20260808 00:37:50.641346 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.629406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47784 (local address 127.1.208.60:35657)
0808 00:37:50.629914 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.629916 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.630223 (+ 307us) server_negotiation.cc:407] Connection header received
0808 00:37:50.630316 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.630317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.630329 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.630368 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.632331 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.633024 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.635725 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.636037 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.636078 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.638159 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.638160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.638160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.638167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.639861 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.640268 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.640270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.640270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.640332 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.640601 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.640602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.640602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.640749 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.640831 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.641185 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.641250 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":77,"threads_started":1}
I20260808 00:37:50.641393 14416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.629113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.1.208.62:40611)
0808 00:37:50.630041 (+ 928us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.630043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.630050 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:50.630445 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.630446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.630458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.635076 (+ 4618us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.635113 (+ 37us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.635829 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.636896 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.637157 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.637215 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.638534 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.638535 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.638535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.638542 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.640309 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.640657 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.640658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.640658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.640713 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.640908 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.640909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.640909 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.641034 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.641117 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.641213 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.641282 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":109,"threads_started":1}
I20260808 00:37:50.642774 12364 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:50.644474 12364 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:50.652557 14434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36471 (local address 127.0.0.1:45336)
0808 00:37:50.648829 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.648840 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.648896 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.649171 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.649172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.649173 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.649201 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.649202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650595 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.650596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.651544 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.651547 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.652283 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.652284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.652341 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.652346 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.652388 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.652434 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":173,"thread_start_us":163,"threads_started":1}
I20260808 00:37:50.652640 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45336 (local address 127.1.208.2:36471)
0808 00:37:50.649006 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.649010 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.649019 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:50.649083 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.649085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.649100 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.649171 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.649249 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650504 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.651612 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.652201 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.652275 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.652444 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.652473 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.652513 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":146,"threads_started":1}
I20260808 00:37:50.652980 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38391 (local address 127.0.0.1:58616)
0808 00:37:50.648632 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.648642 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.648683 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.649148 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.649149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.649151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.649196 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.649198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650559 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.650560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.651841 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.651844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.652705 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.652706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.652788 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.652793 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.652822 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.652858 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":51,"thread_start_us":162,"threads_started":1}
I20260808 00:37:50.653470 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43261 (local address 127.0.0.1:34248)
0808 00:37:50.648328 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.648338 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.648376 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.649472 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.649473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.649475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.649513 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.649514 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650935 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.650936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.651973 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.651975 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.653209 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.653210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.653271 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.653274 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.653363 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.653405 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 00:37:50.653692 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.1.208.3:38391)
0808 00:37:50.648983 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.648988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.648999 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:50.649038 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.649040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.649056 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.649140 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.649273 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650461 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.651947 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.652606 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.653281 (+ 675us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.653487 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.653516 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.653558 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":121,"threads_started":1}
I20260808 00:37:50.653915 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.648460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34248 (local address 127.1.208.1:43261)
0808 00:37:50.649259 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.649264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.649273 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:50.649350 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.649352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.649369 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.649578 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.649633 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.650864 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.652048 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.653111 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.653202 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.653482 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.653516 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.653559 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":158,"threads_started":1}
I20260808 00:37:50.654100 14075 tablet_service.cc:1467] Processing CreateTablet for tablet 7145de1379234a7f83fdaf1b259ae97f (DEFAULT_TABLE table=test-table [id=0f3876b279724a99b669f6fded399b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:50.654482 14075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7145de1379234a7f83fdaf1b259ae97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:50.654750 14285 tablet_service.cc:1467] Processing CreateTablet for tablet 7145de1379234a7f83fdaf1b259ae97f (DEFAULT_TABLE table=test-table [id=0f3876b279724a99b669f6fded399b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:50.654745 13841 tablet_service.cc:1467] Processing CreateTablet for tablet 7145de1379234a7f83fdaf1b259ae97f (DEFAULT_TABLE table=test-table [id=0f3876b279724a99b669f6fded399b4d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:50.655138 13841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7145de1379234a7f83fdaf1b259ae97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:50.655138 14285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7145de1379234a7f83fdaf1b259ae97f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:50.658850 14438 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Bootstrap starting.
I20260808 00:37:50.660434 14440 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Bootstrap starting.
I20260808 00:37:50.660665 14438 tablet_bootstrap.cc:654] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:50.661494 14442 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Bootstrap starting.
I20260808 00:37:50.661738 14438 log.cc:824] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:50.661851 14440 tablet_bootstrap.cc:654] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:50.662817 14440 log.cc:824] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:50.662890 14438 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: No bootstrap required, opened a new log
I20260808 00:37:50.662956 14442 tablet_bootstrap.cc:654] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:50.662966 14438 ts_tablet_manager.cc:1397] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260808 00:37:50.663882 14440 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: No bootstrap required, opened a new log
I20260808 00:37:50.663923 14442 log.cc:824] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:50.663960 14440 ts_tablet_manager.cc:1397] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260808 00:37:50.664683 14442 tablet_bootstrap.cc:492] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: No bootstrap required, opened a new log
I20260808 00:37:50.664741 14442 ts_tablet_manager.cc:1397] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:37:50.667328 14438 raft_consensus.cc:348] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.667511 14438 raft_consensus.cc:374] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:50.667552 14438 raft_consensus.cc:729] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1301b35a4920489697cf5818c7237019, State: Initialized, Role: FOLLOWER
I20260808 00:37:50.667692 14438 consensus_queue.cc:260] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.668078 14438 ts_tablet_manager.cc:1428] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260808 00:37:50.668059 14442 raft_consensus.cc:348] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.668190 14178 heartbeater.cc:503] Master 127.1.208.62:40611 was elected leader, sending a full tablet report...
I20260808 00:37:50.668210 14442 raft_consensus.cc:374] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:50.668242 14442 raft_consensus.cc:729] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 983133ad86d1435eb7eec17b72b76900, State: Initialized, Role: FOLLOWER
I20260808 00:37:50.668409 14442 consensus_queue.cc:260] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.668474 14440 raft_consensus.cc:348] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.668618 14440 raft_consensus.cc:374] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:50.668648 14440 raft_consensus.cc:729] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3650e2a50145eb9590c3bb2971f133, State: Initialized, Role: FOLLOWER
I20260808 00:37:50.668748 14440 consensus_queue.cc:260] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.668799 14442 ts_tablet_manager.cc:1428] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:37:50.669065 14440 ts_tablet_manager.cc:1428] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 00:37:50.669245 14393 heartbeater.cc:503] Master 127.1.208.62:40611 was elected leader, sending a full tablet report...
I20260808 00:37:50.669706 13963 heartbeater.cc:503] Master 127.1.208.62:40611 was elected leader, sending a full tablet report...
W20260808 00:37:50.719012 14180 tablet.cc:2369] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:50.769443 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.766951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.1.208.60:35657)
0808 00:37:50.767009 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.767011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.767020 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:50.767051 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.767052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.767077 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.767136 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.767265 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.767752 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.768696 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.769103 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.769167 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.769276 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.769318 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.769363 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:37:50.769451 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.766866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.0.0.1:47796)
0808 00:37:50.766923 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.766935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.766994 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.767145 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.767146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.767148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.767209 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.767211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.767823 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.767823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.768599 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.768602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.769180 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.769182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.769244 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.769248 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.769320 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.769366 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:37:50.769887 13675 raft_consensus.cc:1203] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260808 00:37:50.770781 14456 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c25e8d957afb476aa914c320c4fb374b. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.771027 14456 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: LEARNER
W20260808 00:37:50.823558 13967 tablet.cc:2369] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:50.844309 14453 raft_consensus.cc:482] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:50.844452 14453 raft_consensus.cc:504] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.844899 14453 leader_election.cc:288] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261), 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:50.848891 14435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.845308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:58061 (local address 127.1.208.1:43261)
0808 00:37:50.845376 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.845378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.845386 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:50.845422 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.845423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.845433 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.845483 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.845744 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.846531 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.847926 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.848491 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.848547 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.848559 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.848783 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.848822 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:37:50.848929 14406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.845089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36471 (local address 127.1.208.3:37027)
0808 00:37:50.845149 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.845162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.845212 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.845450 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.845451 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.845452 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.845678 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.845679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.846580 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.846581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.847852 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.847854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.848557 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.848558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.848767 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.848770 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.848816 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.848860 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:37:50.848941 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.845220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:37027 (local address 127.1.208.2:36471)
0808 00:37:50.845318 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.845320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.845328 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:50.845361 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.845362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.845375 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.845424 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:50.845744 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.846484 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.847926 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.848477 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.848547 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.848559 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.848827 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.848868 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:37:50.848960 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.845099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43261 (local address 127.1.208.3:58061)
0808 00:37:50.845184 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:50.845195 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:50.845235 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:50.845493 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:50.845494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:50.845495 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:50.845674 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.845676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.846596 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.846597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.847844 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:50.847847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.848557 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.848558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:50.848714 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.848719 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:50.848779 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:37:50.848822 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":55}
I20260808 00:37:50.849329 13875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7145de1379234a7f83fdaf1b259ae97f" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3650e2a50145eb9590c3bb2971f133" is_pre_election: true
I20260808 00:37:50.849359 14108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7145de1379234a7f83fdaf1b259ae97f" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1301b35a4920489697cf5818c7237019" is_pre_election: true
I20260808 00:37:50.849489 13875 raft_consensus.cc:2454] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 0.
I20260808 00:37:50.849521 14108 raft_consensus.cc:2454] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 0.
I20260808 00:37:50.849705 14218 leader_election.cc:302] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3650e2a50145eb9590c3bb2971f133, 983133ad86d1435eb7eec17b72b76900; no voters:
I20260808 00:37:50.849880 14453 raft_consensus.cc:2781] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:50.849936 14453 raft_consensus.cc:482] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:50.849959 14453 raft_consensus.cc:3037] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:50.850975 14453 raft_consensus.cc:504] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.851116 14453 leader_election.cc:288] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261), 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:50.851251 13875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7145de1379234a7f83fdaf1b259ae97f" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3650e2a50145eb9590c3bb2971f133"
I20260808 00:37:50.851331 13875 raft_consensus.cc:3037] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:50.851408 14108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7145de1379234a7f83fdaf1b259ae97f" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1301b35a4920489697cf5818c7237019"
I20260808 00:37:50.851498 14108 raft_consensus.cc:3037] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:50.852362 13875 raft_consensus.cc:2454] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 1.
I20260808 00:37:50.852522 14218 leader_election.cc:302] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3650e2a50145eb9590c3bb2971f133, 983133ad86d1435eb7eec17b72b76900; no voters:
I20260808 00:37:50.852615 14453 raft_consensus.cc:2781] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:50.852766 14453 raft_consensus.cc:686] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 1 LEADER]: Becoming Leader. State: Replica: 983133ad86d1435eb7eec17b72b76900, State: Running, Role: LEADER
I20260808 00:37:50.852866 14453 consensus_queue.cc:237] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } }
I20260808 00:37:50.852898 14108 raft_consensus.cc:2454] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 1.
I20260808 00:37:50.853955 12363 catalog_manager.cc:5495] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 983133ad86d1435eb7eec17b72b76900 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "983133ad86d1435eb7eec17b72b76900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } health_report { overall_health: HEALTHY } } }
I20260808 00:37:50.854614 14468 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: NON_VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: true } } }
I20260808 00:37:50.854744 14468 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.855307 13280 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: attempting to promote NON_VOTER fcaa2f6aab1e4bcaa3c56f941e397716 to VOTER
I20260808 00:37:50.855538 13280 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:50.855997 13369 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 00:37:50.856122 13675 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 LEARNER]: Refusing update from remote peer c25e8d957afb476aa914c320c4fb374b: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260808 00:37:50.856298 14468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 00:37:50.856452 14475 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260808 00:37:50.857122 13280 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.857277 13369 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.857254 13675 raft_consensus.cc:2932] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858238 13280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858315 13280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.858261 14475 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858354 14475 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.858354 13280 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858418 13280 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:50.858440 14456 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858510 14456 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:50.858598 14477 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c25e8d957afb476aa914c320c4fb374b" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:50.858685 14477 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:37:50.872229 14403 tablet.cc:2369] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:50.877718 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.875391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.1.208.3:38391)
0808 00:37:50.875447 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.875449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.875458 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:50.875497 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.875497 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.875517 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.875576 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:50.875777 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.876339 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.877018 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.877192 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.877232 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.877434 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:50.877535 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:50.877597 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.877610 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.877628 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:37:50.880024 13875 raft_consensus.cc:1261] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Refusing update from remote peer 983133ad86d1435eb7eec17b72b76900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:50.880076 14108 raft_consensus.cc:1261] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Refusing update from remote peer 983133ad86d1435eb7eec17b72b76900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:50.880430 14465 consensus_queue.cc:1035] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:50.880555 14453 consensus_queue.cc:1035] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:50.882987 14480 mvcc.cc:205] Tried to move back new op lower bound from 7316068232721936384 to 7316068232614035456. Current Snapshot: MvccSnapshot[applied={T|T < 7316068232721936384}]
I20260808 00:37:50.884470 14481 mvcc.cc:205] Tried to move back new op lower bound from 7316068232721936384 to 7316068232614035456. Current Snapshot: MvccSnapshot[applied={T|T < 7316068232721936384}]
I20260808 00:37:50.888345 12364 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:50.888451 12364 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:50.889379 14482 mvcc.cc:205] Tried to move back new op lower bound from 7316068232721936384 to 7316068232614035456. Current Snapshot: MvccSnapshot[applied={T|T < 7316068232721936384 or (T in {7316068232721936384})}]
I20260808 00:37:50.893814 12364 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c25e8d957afb476aa914c320c4fb374b: Sending DeleteTablet for 3 replicas of tablet 7145de1379234a7f83fdaf1b259ae97f
I20260808 00:37:50.894446 14285 tablet_service.cc:1514] Processing DeleteTablet for tablet 7145de1379234a7f83fdaf1b259ae97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:50 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:58616
I20260808 00:37:50.894909 14497 tablet_replica.cc:330] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: stopping tablet replica
I20260808 00:37:50.894997 14075 tablet_service.cc:1514] Processing DeleteTablet for tablet 7145de1379234a7f83fdaf1b259ae97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:50 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45336
I20260808 00:37:50.895181 14497 raft_consensus.cc:2229] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:50.895337 14497 raft_consensus.cc:2258] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:50.895393 14500 tablet_replica.cc:330] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: stopping tablet replica
I20260808 00:37:50.895521 14500 raft_consensus.cc:2229] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:50.895530 13841 tablet_service.cc:1514] Processing DeleteTablet for tablet 7145de1379234a7f83fdaf1b259ae97f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:50 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:34248
I20260808 00:37:50.895640 14500 raft_consensus.cc:2258] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:50.896200 14497 ts_tablet_manager.cc:1905] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:50.896229 14500 ts_tablet_manager.cc:1905] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:50.896840 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 12313
I20260808 00:37:50.916527 14500 ts_tablet_manager.cc:1918] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:50.916664 14500 log.cc:1196] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/7145de1379234a7f83fdaf1b259ae97f
I20260808 00:37:50.916990 14500 ts_tablet_manager.cc:1939] T 7145de1379234a7f83fdaf1b259ae97f P 1301b35a4920489697cf5818c7237019: Deleting consensus metadata
I20260808 00:37:50.919483 14497 ts_tablet_manager.cc:1918] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:50.919656 14497 log.cc:1196] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/7145de1379234a7f83fdaf1b259ae97f
I20260808 00:37:50.920025 14497 ts_tablet_manager.cc:1939] T 7145de1379234a7f83fdaf1b259ae97f P 983133ad86d1435eb7eec17b72b76900: Deleting consensus metadata
W20260808 00:37:50.921960 14218 connection.cc:535] server connection from 127.0.0.1:58616 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:37:50.922230 14002 connection.cc:535] server connection from 127.0.0.1:45336 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:37:50.924252 14501 tablet_replica.cc:330] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: stopping tablet replica
I20260808 00:37:50.924389 14501 raft_consensus.cc:2229] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:50.924522 14501 raft_consensus.cc:2258] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:50.925709 14501 ts_tablet_manager.cc:1905] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:50.927642 14501 ts_tablet_manager.cc:1918] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:50.927732 14501 log.cc:1196] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/7145de1379234a7f83fdaf1b259ae97f
I20260808 00:37:50.928017 14501 ts_tablet_manager.cc:1939] T 7145de1379234a7f83fdaf1b259ae97f P 3c3650e2a50145eb9590c3bb2971f133: Deleting consensus metadata
W20260808 00:37:50.928390 13781 connection.cc:441] server connection from 127.0.0.1:34248 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:34248 (request call id 1) could send its response
I20260808 00:37:50.933555 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.924753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.1.208.61:37213)
0808 00:37:50.924807 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.924809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.924911 (+ 102us) server_negotiation.cc:407] Connection header received
0808 00:37:50.925042 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.925044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.925059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.925088 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.926203 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.926986 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.928264 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.928564 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.928605 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.930512 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.930513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.930513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.930521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.932266 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.932652 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.932653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.932653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.932710 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.932911 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.932912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.932913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.933041 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.933116 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.933363 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.933457 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:37:50.934072 14419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.896131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47810 (local address 127.1.208.60:35657)
0808 00:37:50.896173 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.896175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.921524 (+ 25349us) server_negotiation.cc:407] Connection header received
0808 00:37:50.921688 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.921690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.921705 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.921741 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.922639 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.923314 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.924170 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.924435 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.924508 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.930356 (+ 5848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.930358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.930358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.930366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.932567 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.932961 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.932962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.932963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.933011 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.933225 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.933226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.933226 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.933388 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.933482 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.933861 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.933964 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:37:50.934132 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:50.925033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47820 (local address 127.1.208.60:35657)
0808 00:37:50.925079 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:50.925081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:50.925363 (+ 282us) server_negotiation.cc:407] Connection header received
0808 00:37:50.925455 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:50.925456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:50.925468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:50.925500 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:50.926628 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.927305 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.928349 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:50.928601 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:50.928639 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:50.930614 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:50.930615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:50.930616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:50.930623 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:50.932463 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.932921 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.932922 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.932922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.932984 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:50.933226 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:50.933227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:50.933227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:50.933611 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:50.933706 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:50.933901 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 00:37:50.934016 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":26}
I20260808 00:37:51.885670 14517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:51.885409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:47205)
0808 00:37:51.885526 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:51.885613 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":59,"threads_started":1}
W20260808 00:37:51.886075 13963 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:40611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:51.886854 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:51.886544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:43993)
0808 00:37:51.886730 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:51.886809 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":58,"threads_started":1}
W20260808 00:37:51.887135 14393 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:40611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:51.887655 14519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:51.887409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:38921)
0808 00:37:51.887529 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:51.887606 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
W20260808 00:37:51.887879 14178 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:40611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.391242 14521 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c25e8d957afb476aa914c320c4fb374b)
I20260808 00:37:52.391256 14520 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c25e8d957afb476aa914c320c4fb374b)
I20260808 00:37:52.391435 14521 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:52.391451 14520 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:52.391736 14521 leader_election.cc:288] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61:37213)
I20260808 00:37:52.391739 14520 leader_election.cc:288] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657)
I20260808 00:37:52.392148 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.391883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43094)
0808 00:37:52.392045 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392102 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":51,"threads_started":1}
W20260808 00:37:52.392289 13641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.392328 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.391892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43102)
0808 00:37:52.392219 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392284 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":69,"threads_started":1}
W20260808 00:37:52.392436 13323 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.392845 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.392765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43108)
0808 00:37:52.392799 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392821 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 00:37:52.392880 14525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.392795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43120)
0808 00:37:52.392830 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392852 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 00:37:52.392951 13323 leader_election.cc:334] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
W20260808 00:37:52.393052 13641 leader_election.cc:334] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.394953 14529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.392112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.1.208.61:37213)
0808 00:37:52.392296 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:52.392298 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:52.392308 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:52.392339 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:52.392340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:52.392355 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:52.392438 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:52.392632 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.393364 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394270 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.394646 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394707 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:52.394718 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:52.394832 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 00:37:52.394873 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":58,"threads_started":1}
I20260808 00:37:52.394958 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.391919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37213 (local address 127.0.0.1:50390)
0808 00:37:52.392066 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392077 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:52.392137 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:52.392501 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:52.392502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:52.392505 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:52.392562 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:52.392564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.393443 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.393444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:52.394189 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:52.394191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.394716 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:52.394774 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:52.394777 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:52.394822 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:37:52.394865 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":52,"threads_started":1}
I20260808 00:37:52.395015 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.392116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47826 (local address 127.1.208.60:35657)
0808 00:37:52.392188 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:52.392190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:52.392413 (+ 223us) server_negotiation.cc:407] Connection header received
0808 00:37:52.392440 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:52.392441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:52.392451 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:52.392509 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:52.392688 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.393403 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394309 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.394673 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394732 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:52.394743 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:52.394865 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 00:37:52.394907 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59,"thread_start_us":35,"threads_started":1}
I20260808 00:37:52.395143 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.391917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:35657 (local address 127.0.0.1:47826)
0808 00:37:52.392244 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.392253 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:52.392363 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:52.392558 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:52.392560 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:52.392562 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:52.392613 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:52.392616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.393486 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.393487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:52.394224 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:52.394227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:52.394743 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:52.394744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:52.394807 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:52.394810 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:52.394853 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:37:52.394899 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":57,"threads_started":1}
I20260808 00:37:52.395226 13369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" is_pre_election: true
I20260808 00:37:52.395354 13369 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fcaa2f6aab1e4bcaa3c56f941e397716 in term 1.
I20260808 00:37:52.395520 13640 leader_election.cc:302] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf28297c93f04a05b1b681f4ba84f5ba, fcaa2f6aab1e4bcaa3c56f941e397716; no voters: c25e8d957afb476aa914c320c4fb374b
I20260808 00:37:52.395490 13675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" is_pre_election: true
I20260808 00:37:52.395596 13675 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf28297c93f04a05b1b681f4ba84f5ba in term 1.
I20260808 00:37:52.395637 14521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:37:52.395674 14521 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c25e8d957afb476aa914c320c4fb374b)
I20260808 00:37:52.395694 14521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:37:52.395757 13324 leader_election.cc:302] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf28297c93f04a05b1b681f4ba84f5ba, fcaa2f6aab1e4bcaa3c56f941e397716; no voters: c25e8d957afb476aa914c320c4fb374b
I20260808 00:37:52.395855 14520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 00:37:52.395901 14520 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Starting leader election (detected failure of leader c25e8d957afb476aa914c320c4fb374b)
I20260808 00:37:52.395931 14520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:37:52.396507 14521 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:52.396574 14520 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:52.396631 14521 leader_election.cc:288] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 election: Requested vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), cf28297c93f04a05b1b681f4ba84f5ba (127.1.208.61:37213)
I20260808 00:37:52.396708 14520 leader_election.cc:288] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 election: Requested vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657)
I20260808 00:37:52.396821 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.396731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43132)
0808 00:37:52.396772 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.396796 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:37:52.396870 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.396781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43148)
0808 00:37:52.396823 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.396848 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 00:37:52.396904 13675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
I20260808 00:37:52.396988 13675 raft_consensus.cc:2379] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cf28297c93f04a05b1b681f4ba84f5ba in current term 2: Already voted for candidate fcaa2f6aab1e4bcaa3c56f941e397716 in this term.
I20260808 00:37:52.397074 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.396998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43154)
0808 00:37:52.397029 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.397053 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260808 00:37:52.396874 13369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cf28297c93f04a05b1b681f4ba84f5ba"
I20260808 00:37:52.397130 13369 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate fcaa2f6aab1e4bcaa3c56f941e397716 in current term 2: Already voted for candidate cf28297c93f04a05b1b681f4ba84f5ba in this term.
W20260808 00:37:52.397171 13641 leader_election.cc:334] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.397194 14524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.397101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43166)
0808 00:37:52.397150 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.397174 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:37:52.397310 13640 leader_election.cc:302] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fcaa2f6aab1e4bcaa3c56f941e397716; no voters: c25e8d957afb476aa914c320c4fb374b, cf28297c93f04a05b1b681f4ba84f5ba
W20260808 00:37:52.397284 13323 leader_election.cc:334] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:52.397349 13323 leader_election.cc:302] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf28297c93f04a05b1b681f4ba84f5ba; no voters: c25e8d957afb476aa914c320c4fb374b, fcaa2f6aab1e4bcaa3c56f941e397716
I20260808 00:37:52.397383 14521 raft_consensus.cc:2726] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:37:52.397444 14520 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 00:37:52.886660 14537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.886428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:55195)
0808 00:37:52.886566 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.886611 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260808 00:37:52.887643 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.887460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:48765)
0808 00:37:52.887567 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.887606 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":60,"threads_started":1}
I20260808 00:37:52.888394 14541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:52.888204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:37327)
0808 00:37:52.888315 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:52.888355 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":54,"threads_started":1}
I20260808 00:37:53.887408 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:53.887109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:42833)
0808 00:37:53.887297 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:53.887345 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":79,"threads_started":1}
I20260808 00:37:53.888360 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:53.888098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:51169)
0808 00:37:53.888274 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:53.888312 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":53,"threads_started":1}
I20260808 00:37:53.888962 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:53.888778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:38503)
0808 00:37:53.888886 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:53.888922 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":50,"threads_started":1}
I20260808 00:37:54.539036 14549 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:54.539160 14549 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:54.539361 14549 leader_election.cc:288] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657)
I20260808 00:37:54.539624 13675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" is_pre_election: true
I20260808 00:37:54.539727 13675 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cf28297c93f04a05b1b681f4ba84f5ba in term 2.
I20260808 00:37:54.539736 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.539537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43472)
0808 00:37:54.539652 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.539690 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":52,"threads_started":1}
I20260808 00:37:54.539901 13324 leader_election.cc:302] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cf28297c93f04a05b1b681f4ba84f5ba, fcaa2f6aab1e4bcaa3c56f941e397716; no voters:
I20260808 00:37:54.540027 14549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 00:37:54.540066 14549 raft_consensus.cc:482] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:54.540089 14549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:37:54.540232 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.540119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43482)
0808 00:37:54.540173 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.540206 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:37:54.540313 13323 leader_election.cc:334] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:54.540925 14549 raft_consensus.cc:504] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 3 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:54.541054 14549 leader_election.cc:288] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 election: Requested vote from peers c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611), fcaa2f6aab1e4bcaa3c56f941e397716 (127.1.208.60:35657)
I20260808 00:37:54.541237 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.541147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43484)
0808 00:37:54.541182 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.541210 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260808 00:37:54.541253 13675 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716"
I20260808 00:37:54.541356 13675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:37:54.541595 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.541501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43494)
0808 00:37:54.541544 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.541573 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:37:54.541675 13323 leader_election.cc:334] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
I20260808 00:37:54.542104 13675 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cf28297c93f04a05b1b681f4ba84f5ba in term 3.
I20260808 00:37:54.542272 13324 leader_election.cc:302] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cf28297c93f04a05b1b681f4ba84f5ba, fcaa2f6aab1e4bcaa3c56f941e397716; no voters: c25e8d957afb476aa914c320c4fb374b
I20260808 00:37:54.542377 14549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:37:54.542495 14549 raft_consensus.cc:686] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [term 3 LEADER]: Becoming Leader. State: Replica: cf28297c93f04a05b1b681f4ba84f5ba, State: Running, Role: LEADER
I20260808 00:37:54.542572 14549 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } }
I20260808 00:37:54.542943 14556 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf28297c93f04a05b1b681f4ba84f5ba. Latest consensus state: current_term: 3 leader_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:54.543045 14556 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:54.543234 14558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:54.543649 14558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:54.543706 14558 catalog_manager.cc:1259] Loaded cluster ID: a014654d9b1144e7931c779c0de36b4a
I20260808 00:37:54.543727 14558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:37:54.543877 14558 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:54.543932 14558 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Loaded TSK: 0
I20260808 00:37:54.544024 14558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:37:54.577322 13342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:54.578991 13342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:54.580025 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.579912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43522)
0808 00:37:54.579962 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.579995 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 00:37:54.580090 13675 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [term 3 FOLLOWER]: Refusing update from remote peer cf28297c93f04a05b1b681f4ba84f5ba: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 00:37:54.580370 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.580294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.0.0.1:43526)
0808 00:37:54.580323 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.580351 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 00:37:54.580454 13323 consensus_peers.cc:487] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba -> Peer c25e8d957afb476aa914c320c4fb374b (127.1.208.62:40611): Couldn't send request to peer c25e8d957afb476aa914c320c4fb374b. Status: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 00:37:54.580479 14556 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [LEADER]: Connected to new peer: Peer: permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 00:37:54.581061 14564 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf28297c93f04a05b1b681f4ba84f5ba. Latest consensus state: current_term: 3 leader_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:54.581151 14564 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:54.581805 14556 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:54.581938 14556 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:54.581913 14564 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "c25e8d957afb476aa914c320c4fb374b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40611 } } peers { permanent_uuid: "cf28297c93f04a05b1b681f4ba84f5ba" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37213 } attrs { promote: false } } peers { permanent_uuid: "fcaa2f6aab1e4bcaa3c56f941e397716" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 35657 } attrs { promote: false } } }
I20260808 00:37:54.582010 14564 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcaa2f6aab1e4bcaa3c56f941e397716 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:37:54.587185 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.583731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38391 (local address 127.0.0.1:58638)
0808 00:37:54.583778 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.583789 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:54.584193 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:54.584352 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:54.584353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:54.584355 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:54.584410 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.584412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585123 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.585124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.586259 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.586262 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.586939 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.586940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.587003 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.587007 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:54.587053 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:37:54.587100 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:37:54.587286 14574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.583791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43261 (local address 127.0.0.1:34252)
0808 00:37:54.584032 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.584040 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:54.584103 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:54.584609 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:54.584610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:54.584612 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:54.584655 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.584656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585522 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.585523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.586390 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.586392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.587059 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.587060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.587110 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.587114 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:54.587163 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:54.587194 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":99,"threads_started":1}
I20260808 00:37:54.587438 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.583986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.1.208.3:38391)
0808 00:37:54.584194 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:54.584195 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:54.584204 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:54.584254 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:54.584254 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:54.584265 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:54.584344 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:54.584477 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585056 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.586369 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.586869 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.586929 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.586945 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:54.587310 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 00:37:54.587367 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":62,"threads_started":1}
I20260808 00:37:54.587596 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.583755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:36471 (local address 127.0.0.1:45344)
0808 00:37:54.583928 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.584075 (+ 147us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:54.584115 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:54.584458 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:54.584459 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:54.584462 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:54.584513 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.584515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585491 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.585492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.586589 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:54.586591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.587364 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.587365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:54.587441 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.587446 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:54.587489 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:37:54.587527 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":33,"thread_start_us":120,"threads_started":1}
I20260808 00:37:54.587639 14285 tablet_service.cc:1467] Processing CreateTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 (DEFAULT_TABLE table=test-table [id=14d88855c6b443c9abb8b3105491d931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:54.587785 14285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49572d2f72f147248f2bdf379c4f00e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:54.587821 14577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.583864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.1.208.2:36471)
0808 00:37:54.584297 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:54.584300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:54.584307 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:54.584333 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:54.584333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:54.584344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:54.584405 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:54.584597 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585389 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.586666 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.587239 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.587310 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.587326 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:54.587697 (+ 371us) server_negotiation.cc:299] Negotiation successful
0808 00:37:54.587750 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":79,"threads_started":1}
I20260808 00:37:54.587899 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.584022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.1.208.1:43261)
0808 00:37:54.584490 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:54.584493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:54.584501 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:54.584526 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:54.584527 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:54.584536 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:54.584601 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:54.584716 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.585464 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.586473 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.586980 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.587051 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.587063 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:54.587175 (+ 112us) server_negotiation.cc:299] Negotiation successful
0808 00:37:54.587217 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":71,"threads_started":1}
I20260808 00:37:54.588120 14075 tablet_service.cc:1467] Processing CreateTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 (DEFAULT_TABLE table=test-table [id=14d88855c6b443c9abb8b3105491d931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:54.588255 14075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49572d2f72f147248f2bdf379c4f00e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:54.588219 13841 tablet_service.cc:1467] Processing CreateTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 (DEFAULT_TABLE table=test-table [id=14d88855c6b443c9abb8b3105491d931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:54.588347 13841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49572d2f72f147248f2bdf379c4f00e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:54.589084 14582 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Bootstrap starting.
I20260808 00:37:54.589546 14582 tablet_bootstrap.cc:654] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:54.589792 14584 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: Bootstrap starting.
I20260808 00:37:54.590188 14582 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: No bootstrap required, opened a new log
I20260808 00:37:54.590294 14582 ts_tablet_manager.cc:1397] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:54.590435 14582 raft_consensus.cc:348] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.590516 14582 raft_consensus.cc:374] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:54.590510 14584 tablet_bootstrap.cc:654] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:54.590540 14582 raft_consensus.cc:729] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 983133ad86d1435eb7eec17b72b76900, State: Initialized, Role: FOLLOWER
I20260808 00:37:54.590590 14582 consensus_queue.cc:260] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.590744 14582 ts_tablet_manager.cc:1428] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:54.590965 14585 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Bootstrap starting.
I20260808 00:37:54.591212 14584 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: No bootstrap required, opened a new log
I20260808 00:37:54.591317 14584 ts_tablet_manager.cc:1397] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:54.591334 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.590934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:57189)
0808 00:37:54.591252 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.591295 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":81,"threads_started":1}
I20260808 00:37:54.591581 14584 raft_consensus.cc:348] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.591691 14584 raft_consensus.cc:374] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:54.591734 14584 raft_consensus.cc:729] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c3650e2a50145eb9590c3bb2971f133, State: Initialized, Role: FOLLOWER
I20260808 00:37:54.591748 14399 heartbeater.cc:503] Master 127.1.208.61:37213 was elected leader, sending a full tablet report...
I20260808 00:37:54.591756 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.591509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:40183)
0808 00:37:54.591626 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.591729 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":75,"threads_started":1}
I20260808 00:37:54.591792 14584 consensus_queue.cc:260] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.591976 14584 ts_tablet_manager.cc:1428] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:54.592423 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.592182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:40217)
0808 00:37:54.592349 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.592384 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260808 00:37:54.592461 14585 tablet_bootstrap.cc:654] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:54.592685 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.592611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:54727)
0808 00:37:54.592646 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.592665 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260808 00:37:54.592767 13966 heartbeater.cc:503] Master 127.1.208.61:37213 was elected leader, sending a full tablet report...
I20260808 00:37:54.593039 14585 tablet_bootstrap.cc:492] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: No bootstrap required, opened a new log
I20260808 00:37:54.593086 14585 ts_tablet_manager.cc:1397] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:54.593205 14585 raft_consensus.cc:348] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.593266 14585 raft_consensus.cc:374] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:54.593284 14585 raft_consensus.cc:729] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1301b35a4920489697cf5818c7237019, State: Initialized, Role: FOLLOWER
I20260808 00:37:54.593366 14585 consensus_queue.cc:260] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.593544 14585 ts_tablet_manager.cc:1428] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:54.593906 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.593688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:44733)
0808 00:37:54.593831 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.593870 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260808 00:37:54.594347 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.594240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:34831)
0808 00:37:54.594286 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.594318 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:37:54.650017 14587 raft_consensus.cc:482] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:54.650149 14587 raft_consensus.cc:504] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.650357 14587 leader_election.cc:288] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261), 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:54.650609 13875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49572d2f72f147248f2bdf379c4f00e3" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3650e2a50145eb9590c3bb2971f133" is_pre_election: true
I20260808 00:37:54.650615 14108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49572d2f72f147248f2bdf379c4f00e3" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1301b35a4920489697cf5818c7237019" is_pre_election: true
I20260808 00:37:54.650707 13875 raft_consensus.cc:2454] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 0.
I20260808 00:37:54.650724 14108 raft_consensus.cc:2454] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 0.
I20260808 00:37:54.650887 14218 leader_election.cc:302] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3650e2a50145eb9590c3bb2971f133, 983133ad86d1435eb7eec17b72b76900; no voters:
I20260808 00:37:54.651011 14587 raft_consensus.cc:2781] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:54.651046 14587 raft_consensus.cc:482] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:54.651065 14587 raft_consensus.cc:3037] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:54.651858 14587 raft_consensus.cc:504] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.652002 14587 leader_election.cc:288] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 election: Requested vote from peers 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261), 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471)
I20260808 00:37:54.652197 14108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49572d2f72f147248f2bdf379c4f00e3" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1301b35a4920489697cf5818c7237019"
I20260808 00:37:54.652186 13875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49572d2f72f147248f2bdf379c4f00e3" candidate_uuid: "983133ad86d1435eb7eec17b72b76900" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c3650e2a50145eb9590c3bb2971f133"
I20260808 00:37:54.652278 14108 raft_consensus.cc:3037] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:54.652278 13875 raft_consensus.cc:3037] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:54.653087 14108 raft_consensus.cc:2454] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 1.
I20260808 00:37:54.653086 13875 raft_consensus.cc:2454] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 983133ad86d1435eb7eec17b72b76900 in term 1.
I20260808 00:37:54.653333 14218 leader_election.cc:302] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c3650e2a50145eb9590c3bb2971f133, 983133ad86d1435eb7eec17b72b76900; no voters:
I20260808 00:37:54.653440 14587 raft_consensus.cc:2781] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:54.653605 14587 raft_consensus.cc:686] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 1 LEADER]: Becoming Leader. State: Replica: 983133ad86d1435eb7eec17b72b76900, State: Running, Role: LEADER
I20260808 00:37:54.653651 14587 consensus_queue.cc:237] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } }
I20260808 00:37:54.653991 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.653855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:42613)
0808 00:37:54.653900 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.653953 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:37:54.654304 13344 catalog_manager.cc:5495] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 reported cstate change: term changed from 0 to 1, leader changed from <none> to 983133ad86d1435eb7eec17b72b76900 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "983133ad86d1435eb7eec17b72b76900" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "983133ad86d1435eb7eec17b72b76900" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:54.662168 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.660222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58646 (local address 127.1.208.3:38391)
0808 00:37:54.660262 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:54.660264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:54.660270 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:37:54.660294 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:54.660295 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:54.660306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:54.660363 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:54.660472 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.660941 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.661658 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:54.661852 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:54.661901 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:54.661993 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:54.662018 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:54.662048 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:54.662086 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:37:54.662104 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:54.663118 14108 raft_consensus.cc:1261] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Refusing update from remote peer 983133ad86d1435eb7eec17b72b76900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:54.663126 13875 raft_consensus.cc:1261] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Refusing update from remote peer 983133ad86d1435eb7eec17b72b76900: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:54.663357 14600 consensus_queue.cc:1035] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1301b35a4920489697cf5818c7237019" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 36471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:54.663472 14587 consensus_queue.cc:1035] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c3650e2a50145eb9590c3bb2971f133" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:54.663779 14179 heartbeater.cc:503] Master 127.1.208.61:37213 was elected leader, sending a full tablet report...
I20260808 00:37:54.664099 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.663980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:43149)
0808 00:37:54.664031 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.664064 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:37:54.664953 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.664807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:36865)
0808 00:37:54.664889 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.664919 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 00:37:54.665566 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.665471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:40583)
0808 00:37:54.665506 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.665535 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 00:37:54.666090 14594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.665991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.1:59109)
0808 00:37:54.666028 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.666060 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 00:37:54.666358 14589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.666271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.3:53415)
0808 00:37:54.666309 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.666332 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:37:54.666395 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.666312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40611 (local address 127.1.208.2:37799)
0808 00:37:54.666348 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.666368 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:40611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 00:37:54.667901 13344 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:54.668007 13344 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50376:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:54.669469 13344 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cf28297c93f04a05b1b681f4ba84f5ba: Sending DeleteTablet for 3 replicas of tablet 49572d2f72f147248f2bdf379c4f00e3
I20260808 00:37:54.669759 14285 tablet_service.cc:1514] Processing DeleteTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:54 UTC) from {username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:58638
I20260808 00:37:54.669796 14075 tablet_service.cc:1514] Processing DeleteTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:54 UTC) from {username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:45344
I20260808 00:37:54.669834 13841 tablet_service.cc:1514] Processing DeleteTablet for tablet 49572d2f72f147248f2bdf379c4f00e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:54 UTC) from {username='oryx', principal='oryx/127.1.208.61@KRBTEST.COM'} at 127.0.0.1:34252
I20260808 00:37:54.669971 14629 tablet_replica.cc:330] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: stopping tablet replica
I20260808 00:37:54.670033 14629 raft_consensus.cc:2229] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:54.670032 14631 tablet_replica.cc:330] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: stopping tablet replica
I20260808 00:37:54.670083 14631 raft_consensus.cc:2229] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:54.670099 14629 raft_consensus.cc:2258] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:54.670125 14631 raft_consensus.cc:2258] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:54.670236 14633 tablet_replica.cc:330] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133: stopping tablet replica
I20260808 00:37:54.670289 14633 raft_consensus.cc:2229] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:54.670301 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 13604
I20260808 00:37:54.670331 14633 raft_consensus.cc:2258] T 49572d2f72f147248f2bdf379c4f00e3 P 3c3650e2a50145eb9590c3bb2971f133 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:54.670562 14629 ts_tablet_manager.cc:1905] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:54.671178 14631 ts_tablet_manager.cc:1905] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:54.673051 14631 ts_tablet_manager.cc:1918] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:54.673110 14631 log.cc:1196] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/49572d2f72f147248f2bdf379c4f00e3
I20260808 00:37:54.673133 14629 ts_tablet_manager.cc:1918] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:54.673208 14629 log.cc:1196] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/49572d2f72f147248f2bdf379c4f00e3
I20260808 00:37:54.673372 14631 ts_tablet_manager.cc:1939] T 49572d2f72f147248f2bdf379c4f00e3 P 1301b35a4920489697cf5818c7237019: Deleting consensus metadata
I20260808 00:37:54.673532 14629 ts_tablet_manager.cc:1939] T 49572d2f72f147248f2bdf379c4f00e3 P 983133ad86d1435eb7eec17b72b76900: Deleting consensus metadata
I20260808 00:37:54.673804 13326 catalog_manager.cc:4841] TS 1301b35a4920489697cf5818c7237019 (127.1.208.2:36471): tablet 49572d2f72f147248f2bdf379c4f00e3 (table test-table [id=14d88855c6b443c9abb8b3105491d931]) successfully deleted
I20260808 00:37:54.677740 13323 catalog_manager.cc:4841] TS 983133ad86d1435eb7eec17b72b76900 (127.1.208.3:38391): tablet 49572d2f72f147248f2bdf379c4f00e3 (table test-table [id=14d88855c6b443c9abb8b3105491d931]) successfully deleted
I20260808 00:37:54.680071 14575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:54.679955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43261 (local address 127.0.0.1:34254)
0808 00:37:54.680003 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:54.680034 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:43261: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:37:54.680186 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 13974
W20260808 00:37:54.680263 13324 catalog_manager.cc:4584] TS 3c3650e2a50145eb9590c3bb2971f133 (127.1.208.1:43261): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49572d2f72f147248f2bdf379c4f00e3: Network error: Client connection negotiation failed: client connection to 127.1.208.1:43261: connect: Connection refused (error 111)
I20260808 00:37:54.693099 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 14185
I20260808 00:37:54.704547 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 13125
I20260808 00:37:54.715353 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 13436
2026-08-08T00:37:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7005 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:54 dist-test-slave-gjw7 krb5kdc[14637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:54 dist-test-slave-gjw7 krb5kdc[14637](info): set up 2 sockets
Aug 08 00:37:54 dist-test-slave-gjw7 krb5kdc[14637](info): commencing operation
krb5kdc: starting...
W20260808 00:37:55.914404 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab.
Entry for principal oryx/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab.
I20260808 00:37:56.043946 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40713
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab
--principal=oryx/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:56.128298 14655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:56.128548 14655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:56.128580 14655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:56.129813 14655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:56.129858 14655 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 00:37:56.129875 14655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:56.129892 14655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:56.129908 14655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:56.129925 14655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:56.131506 14655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.62.keytab
--principal=oryx/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:56.131770 14655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:56.131994 14655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:56.136780 14675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:56.136787 14665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.136971 14655 server_base.cc:1034] running on GCE node
W20260808 00:37:56.137135 14668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.137427 14655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:56.137701 14655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:56.138875 14655 hybrid_clock.cc:648] HybridClock initialized: now 1786149476138857 us; error 31 us; skew 500 ppm
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:56.141512 14655 init.cc:377] Logged in from keytab as oryx/127.1.208.62@KRBTEST.COM (short username oryx)
I20260808 00:37:56.142024 14655 webserver.cc:460] Webserver started at http://127.1.208.62:36747/ using document root <none> and password file <none>
I20260808 00:37:56.142257 14655 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:56.142307 14655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:56.142448 14655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:56.143585 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "6dd96e5fd1fc4ea7a698335a34096577"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "427cd9cf3cb655ad44a12f6d81566ffd"
server_key_iv: "0d7ec589233cd939991903fcbfaba836"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.143965 14655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "6dd96e5fd1fc4ea7a698335a34096577"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "427cd9cf3cb655ad44a12f6d81566ffd"
server_key_iv: "0d7ec589233cd939991903fcbfaba836"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.145440 14655 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:56.146214 14685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.146375 14655 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:56.146469 14655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "6dd96e5fd1fc4ea7a698335a34096577"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "427cd9cf3cb655ad44a12f6d81566ffd"
server_key_iv: "0d7ec589233cd939991903fcbfaba836"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.146533 14655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:56.166769 14655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:56.169087 14655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:56.169216 14655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:56.174273 14655 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:40713
I20260808 00:37:56.174386 14800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:40713 every 8 connection(s)
I20260808 00:37:56.174701 14655 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:37:56.175563 14803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.178124 14803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Bootstrap starting.
I20260808 00:37:56.178776 14803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.179026 14803 log.cc:824] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:56.179719 14803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: No bootstrap required, opened a new log
I20260808 00:37:56.179808 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 14655
I20260808 00:37:56.179903 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:37:56.180092 1856 external_mini_cluster.cc:1428] Setting key 6856f3e5169c7f876e8b0547ab7c45d7
I20260808 00:37:56.182432 14803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } }
I20260808 00:37:56.182634 14803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.182677 14803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dd96e5fd1fc4ea7a698335a34096577, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.182811 14803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } }
I20260808 00:37:56.182895 14803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:37:56.182920 14803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:37:56.182969 14803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:56.183713 14803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } }
I20260808 00:37:56.183835 14803 leader_election.cc:302] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dd96e5fd1fc4ea7a698335a34096577; no voters:
I20260808 00:37:56.184026 14803 leader_election.cc:288] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:37:56.184077 14816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:56.184232 14816 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [term 1 LEADER]: Becoming Leader. State: Replica: 6dd96e5fd1fc4ea7a698335a34096577, State: Running, Role: LEADER
I20260808 00:37:56.184315 14803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:37:56.184337 14816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } }
I20260808 00:37:56.184832 14816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dd96e5fd1fc4ea7a698335a34096577. Latest consensus state: current_term: 1 leader_uuid: "6dd96e5fd1fc4ea7a698335a34096577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } } }
I20260808 00:37:56.184926 14818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dd96e5fd1fc4ea7a698335a34096577" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dd96e5fd1fc4ea7a698335a34096577" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40713 } } }
I20260808 00:37:56.184960 14816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:56.184976 14818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577 [sys.catalog]: This master's current role is: LEADER
I20260808 00:37:56.185225 14827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:37:56.185849 14827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:37:56.187736 14827 catalog_manager.cc:1347] Generated new cluster ID: 71c20361f5e54899ae251d5c941ad3c4
I20260808 00:37:56.187799 14827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:56.192543 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.180628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43372 (local address 127.1.208.62:40713)
0808 00:37:56.180812 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.180823 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.180861 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:37:56.182389 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.182399 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.182473 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.182640 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.183364 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.184752 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.185904 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.186502 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.186689 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.189228 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.189233 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.189233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.189247 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.190982 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.191338 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.191342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.191343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.191449 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.191636 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.191637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.191637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.191905 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.192067 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.192168 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.192316 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260808 00:37:56.206979 14827 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:37:56.207624 14827 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:37:56.210598 14827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Generated new TSK 0
I20260808 00:37:56.210775 14827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
I20260808 00:37:56.250730 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40713
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:56.360049 14859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:56.360297 14859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:56.360323 14859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:56.361593 14859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:56.361649 14859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:56.361670 14859 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:56.361687 14859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:37:56.363430 14859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:40713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:56.363708 14859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:56.363976 14859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:56.369315 14871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.369452 14859 server_base.cc:1034] running on GCE node
W20260808 00:37:56.369340 14873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:56.369556 14877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.369803 14859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:56.370100 14859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:56.371299 14859 hybrid_clock.cc:648] HybridClock initialized: now 1786149476371274 us; error 38 us; skew 500 ppm
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:56.374231 14859 init.cc:377] Logged in from keytab as oryx/127.1.208.1@KRBTEST.COM (short username oryx)
I20260808 00:37:56.374775 14859 webserver.cc:460] Webserver started at http://127.1.208.1:35543/ using document root <none> and password file <none>
I20260808 00:37:56.375005 14859 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:56.375061 14859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:56.375203 14859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:56.376386 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "d294bc1640c3231a1a385af86f779a9b"
server_key_iv: "daaeac057c97d050806ad6acb71b1abc"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.376792 14859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "d294bc1640c3231a1a385af86f779a9b"
server_key_iv: "daaeac057c97d050806ad6acb71b1abc"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.378602 14859 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 00:37:56.379638 14893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.379870 14859 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:56.379977 14859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "d294bc1640c3231a1a385af86f779a9b"
server_key_iv: "daaeac057c97d050806ad6acb71b1abc"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.380054 14859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:56.392251 14859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:56.394605 14859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:56.394775 14859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:56.395321 14859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:56.395368 14859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.395401 14859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:56.395417 14859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:56.402599 14859 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39617
I20260808 00:37:56.402920 15094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39617 every 8 connection(s)
I20260808 00:37:56.403079 14859 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:37:56.405181 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.395883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34669 (local address 127.1.208.62:40713)
0808 00:37:56.395950 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.395952 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.396211 (+ 259us) server_negotiation.cc:407] Connection header received
0808 00:37:56.396794 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.396795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.396809 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.396861 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.397721 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.398293 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.399299 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.399513 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.399569 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.402297 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.402299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.402300 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.402312 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.403856 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.404389 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.404390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.404390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.404450 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.404651 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.404652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.404652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.404763 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.404809 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.405037 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.405112 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:37:56.405452 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.395892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.1:34669)
0808 00:37:56.396090 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.396130 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.396685 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.396943 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.396948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.397375 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.397625 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.397632 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.398375 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.398376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.399213 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.399215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.399602 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.399603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.399738 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.400307 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.400317 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.402175 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.403986 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.404015 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.404018 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.404305 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.404500 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.404500 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.404501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.404606 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.404799 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.404802 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.404939 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.405018 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.405178 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":116,"threads_started":1}
I20260808 00:37:56.406064 15095 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40713
I20260808 00:37:56.406160 15095 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:56.406385 15095 heartbeater.cc:511] Master 127.1.208.62:40713 requested a full tablet report, sending...
I20260808 00:37:56.406960 14715 ts_manager.cc:194] Registered new tserver with Master: e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617)
I20260808 00:37:56.407852 14715 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:34669
I20260808 00:37:56.408349 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 14859
I20260808 00:37:56.408452 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:37:56.408653 1856 external_mini_cluster.cc:1428] Setting key f8be963c6ae90930301270d2455db0b1
I20260808 00:37:56.413141 15105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.407077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.1:49225)
0808 00:37:56.407220 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.407233 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.407316 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.407474 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.407475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.407679 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.407761 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.407763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.408588 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.408589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.409281 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.409284 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.409659 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.409659 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.409725 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.410190 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.410194 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.410549 (+ 355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.412120 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.412121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.412121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.412317 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.412554 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.412554 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.412554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.412611 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.412865 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.412866 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.412920 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.412968 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.413057 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":134,"threads_started":1}
I20260808 00:37:56.413185 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.407184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:49225 (local address 127.1.208.62:40713)
0808 00:37:56.407223 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.407225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.407293 (+ 68us) server_negotiation.cc:407] Connection header received
0808 00:37:56.407414 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.407415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.407423 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.407467 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.407823 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.408478 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.409386 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.409598 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.409655 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.410632 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.410634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.410635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.410643 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.412022 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.412433 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.412434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.412435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.412500 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.412667 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.412668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.412668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.412807 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.412874 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.413005 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.413097 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
I20260808 00:37:56.438608 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40713
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:56.527383 15110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:56.527644 15110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:56.527673 15110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:56.529091 15110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:56.529173 15110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:56.529196 15110 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:56.529214 15110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:37:56.531198 15110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:40713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:56.531464 15110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:56.531682 15110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:56.537163 15125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:56.537163 15123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:56.537168 15122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.537417 15110 server_base.cc:1034] running on GCE node
I20260808 00:37:56.537655 15110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:56.537948 15110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:56.539113 15110 hybrid_clock.cc:648] HybridClock initialized: now 1786149476539096 us; error 34 us; skew 500 ppm
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:56.541983 15110 init.cc:377] Logged in from keytab as oryx/127.1.208.2@KRBTEST.COM (short username oryx)
I20260808 00:37:56.542480 15110 webserver.cc:460] Webserver started at http://127.1.208.2:46851/ using document root <none> and password file <none>
I20260808 00:37:56.542680 15110 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:56.542722 15110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:56.542827 15110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:56.543957 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "c0f278b79b4f4b4bbd719fe46c264281"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "f27538b75b862314256985b96fcc406e"
server_key_iv: "cf1946163960bbf5debd5fad2e6fcbe6"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.544301 15110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "c0f278b79b4f4b4bbd719fe46c264281"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "f27538b75b862314256985b96fcc406e"
server_key_iv: "cf1946163960bbf5debd5fad2e6fcbe6"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.545799 15110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:37:56.546597 15141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.546765 15110 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:56.546864 15110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "c0f278b79b4f4b4bbd719fe46c264281"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "f27538b75b862314256985b96fcc406e"
server_key_iv: "cf1946163960bbf5debd5fad2e6fcbe6"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.546934 15110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:56.558450 15110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:56.561267 15110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:56.561487 15110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:56.562259 15110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:56.562316 15110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.562350 15110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:56.562377 15110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.569686 15110 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:41697
I20260808 00:37:56.569833 15358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:41697 every 8 connection(s)
I20260808 00:37:56.570151 15110 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:56.574517 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.563369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:58147 (local address 127.1.208.62:40713)
0808 00:37:56.563442 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.563444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.563742 (+ 298us) server_negotiation.cc:407] Connection header received
0808 00:37:56.564469 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.564472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.564488 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.564577 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.565653 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.566437 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.568190 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.568547 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.568645 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.571612 (+ 2967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.571614 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.571614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.571623 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.573149 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.573641 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.573642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.573642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.573697 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.573909 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.573910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.573910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.574021 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.574067 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.574374 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.574445 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:37:56.574687 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.563328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.2:58147)
0808 00:37:56.563600 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.563636 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.564337 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.564672 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.564677 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.565117 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.565501 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.565512 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.566593 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.566595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.568040 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.568045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.568670 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.568671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.568879 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.569644 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.569656 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.571496 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.573246 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.573257 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.573260 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.573573 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.573747 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.573748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.573748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.573860 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.574064 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.574067 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.574202 (+ 135us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.574320 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.574470 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
I20260808 00:37:56.575222 15360 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40713
I20260808 00:37:56.575323 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 15110
I20260808 00:37:56.575349 15360 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:56.575430 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:37:56.575585 15360 heartbeater.cc:511] Master 127.1.208.62:40713 requested a full tablet report, sending...
I20260808 00:37:56.575646 1856 external_mini_cluster.cc:1428] Setting key d85f129d71ac093e0f43af9345e66a44
I20260808 00:37:56.576140 14714 ts_manager.cc:194] Registered new tserver with Master: c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697)
I20260808 00:37:56.576622 14714 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:58147
I20260808 00:37:56.583698 15368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.576261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.2:58961)
0808 00:37:56.576483 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.576493 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.576538 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.577500 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.577502 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.577751 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.577844 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.577847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.578680 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.578681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.579475 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.579479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.579852 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.579853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.579918 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.580383 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.580388 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.580733 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.582478 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.582480 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.582481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.582671 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.582944 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.582945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.582945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.583004 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.583383 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.583384 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.583446 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.583498 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.583607 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260808 00:37:56.583747 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.576457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:58961 (local address 127.1.208.62:40713)
0808 00:37:56.577393 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.577395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.577404 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:56.577427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.577427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.577439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.577487 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.577933 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.578573 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.579570 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.579779 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.579848 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.580827 (+ 979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.580828 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.580829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.580847 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.582375 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.582795 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.582797 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.582797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.582875 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.583059 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.583060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.583060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.583314 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.583398 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.583545 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.583638 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":904}
WARNING: no policy specified for oryx/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
I20260808 00:37:56.606765 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40713
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43201
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:56.695029 15372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:56.695290 15372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:56.695329 15372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:56.696619 15372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:56.696676 15372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:56.696715 15372 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 00:37:56.696744 15372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:37:56.698527 15372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43201
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.208.62:40713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:56.698843 15372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:56.699080 15372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:56.704638 15390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.704700 15372 server_base.cc:1034] running on GCE node
W20260808 00:37:56.704641 15381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:56.704643 15387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:56.705062 15372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:56.705400 15372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:56.706560 15372 hybrid_clock.cc:648] HybridClock initialized: now 1786149476706548 us; error 31 us; skew 500 ppm
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:37:56.709265 15372 init.cc:377] Logged in from keytab as oryx/127.1.208.3@KRBTEST.COM (short username oryx)
I20260808 00:37:56.709805 15372 webserver.cc:460] Webserver started at http://127.1.208.3:32855/ using document root <none> and password file <none>
I20260808 00:37:56.710011 15372 fs_manager.cc:359] Metadata directory not provided
I20260808 00:37:56.710052 15372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:37:56.710156 15372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:37:56.711306 15372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "8bde4c1fa7274a968a648c642d469404"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "76a3706055c8d7ce98ffec92dcaa0c46"
server_key_iv: "487421e2371f3477674c7774132e4972"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.711654 15372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "8bde4c1fa7274a968a648c642d469404"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "76a3706055c8d7ce98ffec92dcaa0c46"
server_key_iv: "487421e2371f3477674c7774132e4972"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.713056 15372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:37:56.713955 15404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.714144 15372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:37:56.714241 15372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "8bde4c1fa7274a968a648c642d469404"
format_stamp: "Formatted at 2026-08-08 00:37:56 on dist-test-slave-gjw7"
server_key: "76a3706055c8d7ce98ffec92dcaa0c46"
server_key_iv: "487421e2371f3477674c7774132e4972"
server_key_version: "encryptionkey@0"
I20260808 00:37:56.714318 15372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:37:56.734261 15372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:37:56.736720 15372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:37:56.736891 15372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:37:56.737571 15372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:37:56.737620 15372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.737653 15372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:37:56.737677 15372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:56.745169 15372 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:42701
I20260808 00:37:56.745230 15615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:42701 every 8 connection(s)
I20260808 00:37:56.745644 15372 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:37:56.747972 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.738127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:52187 (local address 127.1.208.62:40713)
0808 00:37:56.738177 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.738179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.738463 (+ 284us) server_negotiation.cc:407] Connection header received
0808 00:37:56.739027 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.739029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.739043 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.739106 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.740107 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.740924 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.742051 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.742272 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.742336 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.745141 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.745144 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.745144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.745155 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.746722 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.747141 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.747142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.747143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.747205 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.747418 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.747418 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.747419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.747532 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.747581 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.747832 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.747902 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:37:56.748175 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.738127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.3:52187)
0808 00:37:56.738326 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.738368 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.738915 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.739238 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.739244 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.739654 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.739987 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.739995 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.741028 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.741029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.741953 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.741957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.742370 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.742371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.742509 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.743121 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.743142 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.745007 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.746824 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.746831 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.746833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.747076 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.747253 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.747253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.747253 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.747374 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.747579 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.747583 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.747715 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.747816 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.747957 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":106,"threads_started":1}
I20260808 00:37:56.748637 15616 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40713
I20260808 00:37:56.748728 15616 heartbeater.cc:464] Registering TS with master...
I20260808 00:37:56.748937 15616 heartbeater.cc:511] Master 127.1.208.62:40713 requested a full tablet report, sending...
I20260808 00:37:56.749410 14715 ts_manager.cc:194] Registered new tserver with Master: 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701)
I20260808 00:37:56.749894 14715 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:52187
I20260808 00:37:56.753778 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 15372
I20260808 00:37:56.753901 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:37:56.754082 1856 external_mini_cluster.cc:1428] Setting key 5c895a4a7fe2fde4b2d5c6b8f680266c
I20260808 00:37:56.755023 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:37:56.755522 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.749857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:47717 (local address 127.1.208.62:40713)
0808 00:37:56.749903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.749905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.750063 (+ 158us) server_negotiation.cc:407] Connection header received
0808 00:37:56.750123 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.750124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.750134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.750177 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.750543 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.751075 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.751776 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.751982 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.752040 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.752952 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.752954 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.752954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.752962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.754400 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.754770 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.754771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.754772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.754836 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.755006 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.755006 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.755007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.755136 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.755195 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.755337 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.755423 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:37:56.755538 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.749812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.1.208.3:47717)
0808 00:37:56.750002 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.750013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.750077 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.750181 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.750182 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.750395 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:56.750465 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.750467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.751166 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.751168 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.751699 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.751701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.752046 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.752047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.752116 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.752542 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:56.752547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:56.752867 (+ 320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:56.754479 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.754480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.754481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.754705 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.754890 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:56.754891 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:56.754891 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:56.754955 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:56.755196 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:56.755197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:56.755251 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.755306 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.755425 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":77,"threads_started":1}
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:37:56.779683 14815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:56.773570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43376 (local address 127.1.208.62:40713)
0808 00:37:56.773639 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.773641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.773651 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:56.773717 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.773718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.773728 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.773781 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.774483 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.775013 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.775953 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.776148 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.776205 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.779591 (+ 3386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43376: BlockingRecv error: recv got EOF from 127.0.0.1:43376 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 00:37:56 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:56.787339 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.780408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43380 (local address 127.1.208.62:40713)
0808 00:37:56.780481 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.780483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.780553 (+ 70us) server_negotiation.cc:407] Connection header received
0808 00:37:56.780624 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.780625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.780636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.780683 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:56.781165 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.781730 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.782432 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.782629 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.782687 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.784995 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:56.784997 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:56.784997 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:56.785007 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:56.786417 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.786755 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.786756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.786756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.786807 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:56.786945 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:56.786945 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:56.786945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:56.787051 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:56.787144 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.787199 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.787264 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 00:37:56.788982 14715 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:56.790776 14715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:56.797657 15654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:42701 (local address 127.0.0.1:42876)
0808 00:37:56.793887 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.793897 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.793949 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.794467 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.794468 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.794470 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:56.794503 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.794504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795671 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.795672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.796561 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.796564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797323 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.797409 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797429 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.797506 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.797563 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260808 00:37:56.797669 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.1.208.3:42701)
0808 00:37:56.794106 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.794109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.794118 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:56.794335 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.794344 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.794377 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.794462 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:56.794562 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795584 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.796667 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797219 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797324 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797429 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.797524 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.797562 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":101,"threads_started":1}
I20260808 00:37:56.797991 15658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41697 (local address 127.0.0.1:44818)
0808 00:37:56.793761 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.793791 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.793938 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.794243 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.794244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.794246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:56.794306 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.794307 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795244 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.795245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.796467 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.796471 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797644 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.797711 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797714 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.797875 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.797920 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":17,"thread_start_us":105,"threads_started":1}
I20260808 00:37:56.797987 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59698 (local address 127.1.208.1:39617)
0808 00:37:56.793659 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.793664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.793876 (+ 212us) server_negotiation.cc:407] Connection header received
0808 00:37:56.794026 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.794030 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.794049 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.794101 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:56.794387 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795536 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.796778 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797511 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797567 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797635 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.797818 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.797866 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260808 00:37:56.798255 15653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39617 (local address 127.0.0.1:59698)
0808 00:37:56.793737 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.793790 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.793880 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.794205 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.794209 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.794212 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:56.794290 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.794293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795631 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.795632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.796695 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.796697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797664 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.797729 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797732 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.798149 (+ 417us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.798195 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":130,"threads_started":1}
I20260808 00:37:56.798681 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.793518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44818 (local address 127.1.208.2:41697)
0808 00:37:56.793715 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.793718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.793905 (+ 187us) server_negotiation.cc:407] Connection header received
0808 00:37:56.794137 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.794140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.794158 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.794239 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:56.794387 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.795161 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.796562 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.797444 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.797516 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.797601 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.798181 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.798231 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260808 00:37:56.798688 15487 tablet_service.cc:1467] Processing CreateTablet for tablet b22808ae58d64849882430a20e047638 (DEFAULT_TABLE table=test-table [id=dd622c723b30469fa702277430f166f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.798995 14979 tablet_service.cc:1467] Processing CreateTablet for tablet b22808ae58d64849882430a20e047638 (DEFAULT_TABLE table=test-table [id=dd622c723b30469fa702277430f166f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.799134 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22808ae58d64849882430a20e047638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.799361 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22808ae58d64849882430a20e047638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.802011 15224 tablet_service.cc:1467] Processing CreateTablet for tablet b22808ae58d64849882430a20e047638 (DEFAULT_TABLE table=test-table [id=dd622c723b30469fa702277430f166f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.802469 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22808ae58d64849882430a20e047638. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.803316 15667 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Bootstrap starting.
I20260808 00:37:56.803946 15667 tablet_bootstrap.cc:654] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.804227 15667 log.cc:824] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:56.805070 15667 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: No bootstrap required, opened a new log
I20260808 00:37:56.805138 15667 ts_tablet_manager.cc:1397] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:56.806217 15671 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Bootstrap starting.
I20260808 00:37:56.806823 15667 raft_consensus.cc:348] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.806993 15667 raft_consensus.cc:374] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.807035 15667 raft_consensus.cc:729] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.807037 15671 tablet_bootstrap.cc:654] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.807160 15667 consensus_queue.cc:260] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.807395 15671 log.cc:824] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:56.807473 15667 ts_tablet_manager.cc:1428] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:37:56.807542 15616 heartbeater.cc:503] Master 127.1.208.62:40713 was elected leader, sending a full tablet report...
I20260808 00:37:56.808254 15671 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: No bootstrap required, opened a new log
I20260808 00:37:56.808336 15671 ts_tablet_manager.cc:1397] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:56.808666 15679 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Bootstrap starting.
I20260808 00:37:56.809219 15679 tablet_bootstrap.cc:654] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.809547 15679 log.cc:824] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Log is configured to *not* fsync() on all Append() calls
I20260808 00:37:56.810250 15679 tablet_bootstrap.cc:492] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: No bootstrap required, opened a new log
I20260808 00:37:56.810316 15679 ts_tablet_manager.cc:1397] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:56.810213 15671 raft_consensus.cc:348] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.810362 15671 raft_consensus.cc:374] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.810410 15671 raft_consensus.cc:729] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.810535 15671 consensus_queue.cc:260] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.810801 15671 ts_tablet_manager.cc:1428] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:37:56.810876 15360 heartbeater.cc:503] Master 127.1.208.62:40713 was elected leader, sending a full tablet report...
I20260808 00:37:56.812146 15679 raft_consensus.cc:348] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.812296 15679 raft_consensus.cc:374] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.812330 15679 raft_consensus.cc:729] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e49d816b847a09f4c5b45c6ca32fa, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.812459 15679 consensus_queue.cc:260] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.812716 15679 ts_tablet_manager.cc:1428] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:37:56.812780 15095 heartbeater.cc:503] Master 127.1.208.62:40713 was elected leader, sending a full tablet report...
W20260808 00:37:56.820842 15361 tablet.cc:2369] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:37:56.904511 15097 tablet.cc:2369] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:37:56.955425 15684 raft_consensus.cc:482] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:56.955574 15684 raft_consensus.cc:504] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.955957 15684 leader_election.cc:288] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701), e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617)
I20260808 00:37:56.959179 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.956326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:44295 (local address 127.1.208.1:39617)
0808 00:37:56.956374 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.956376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.956402 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:37:56.956433 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.956434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.956447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.956498 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:56.956820 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.957419 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958435 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.958829 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958889 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.958904 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.959061 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.959102 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:56.959179 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.956346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:50999 (local address 127.1.208.3:42701)
0808 00:37:56.956418 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.956420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.956611 (+ 191us) server_negotiation.cc:407] Connection header received
0808 00:37:56.956643 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.956644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.956658 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.956718 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:56.957008 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.957557 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958481 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.958836 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958900 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.958911 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.959055 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.959096 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:37:56.959198 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.956228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39617 (local address 127.1.208.2:44295)
0808 00:37:56.956297 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.956309 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.956369 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.956609 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.956611 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.956614 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:56.956707 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.956710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.957501 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.957502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.958334 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.958337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.958911 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.958991 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.958994 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.959043 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.959089 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 00:37:56.959201 15686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.956252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:42701 (local address 127.1.208.2:50999)
0808 00:37:56.956498 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:56.956508 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:56.956558 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:56.956769 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:56.956771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:56.956774 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:56.956868 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.956870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.957644 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.957645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.958385 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:56.958387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.958916 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.958917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:56.958990 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.958994 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:56.959040 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:37:56.959088 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":186,"threads_started":1}
I20260808 00:37:56.959636 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22808ae58d64849882430a20e047638" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" is_pre_election: true
I20260808 00:37:56.959684 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22808ae58d64849882430a20e047638" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404" is_pre_election: true
I20260808 00:37:56.959817 15021 raft_consensus.cc:2454] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:56.959856 15534 raft_consensus.cc:2454] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:56.960101 15142 leader_election.cc:302] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f278b79b4f4b4bbd719fe46c264281, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:56.960268 15684 raft_consensus.cc:2781] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:56.960336 15684 raft_consensus.cc:482] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:56.960362 15684 raft_consensus.cc:3037] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:56.961472 15684 raft_consensus.cc:504] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.961640 15684 leader_election.cc:288] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Requested vote from peers 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701), e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617)
I20260808 00:37:56.961836 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22808ae58d64849882430a20e047638" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404"
I20260808 00:37:56.961882 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22808ae58d64849882430a20e047638" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
I20260808 00:37:56.961944 15534 raft_consensus.cc:3037] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:56.961963 15021 raft_consensus.cc:3037] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:56.962868 15534 raft_consensus.cc:2454] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:56.962868 15021 raft_consensus.cc:2454] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:56.963104 15145 leader_election.cc:302] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, c0f278b79b4f4b4bbd719fe46c264281; no voters:
I20260808 00:37:56.963203 15684 raft_consensus.cc:2781] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:56.963357 15684 raft_consensus.cc:686] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 LEADER]: Becoming Leader. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Running, Role: LEADER
I20260808 00:37:56.963490 15684 consensus_queue.cc:237] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:56.964491 14715 catalog_manager.cc:5495] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:56.979060 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:56.976536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44832 (local address 127.1.208.2:41697)
0808 00:37:56.976597 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:56.976599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:56.976612 (+ 13us) server_negotiation.cc:407] Connection header received
0808 00:37:56.976659 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:56.976659 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:56.976683 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:56.976751 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:56.976929 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.977512 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.978301 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:56.978512 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:56.978565 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:56.978752 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:56.978880 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:56.978953 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:56.978965 (+ 12us) server_negotiation.cc:299] Negotiation successful
0808 00:37:56.978989 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:37:56.981141 15021 raft_consensus.cc:1261] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:56.981140 15534 raft_consensus.cc:1261] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:56.981531 15688 consensus_queue.cc:1035] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:56.981675 15684 consensus_queue.cc:1035] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:56.984756 15699 mvcc.cc:205] Tried to move back new op lower bound from 7316068257711792128 to 7316068257642991616. Current Snapshot: MvccSnapshot[applied={T|T < 7316068257711792128}]
I20260808 00:37:56.985090 15700 mvcc.cc:205] Tried to move back new op lower bound from 7316068257711792128 to 7316068257642991616. Current Snapshot: MvccSnapshot[applied={T|T < 7316068257711792128}]
I20260808 00:37:56.987844 14714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:56.987955 14714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:56.988886 14714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Sending DeleteTablet for 3 replicas of tablet b22808ae58d64849882430a20e047638
I20260808 00:37:56.989233 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet b22808ae58d64849882430a20e047638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:56 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 00:37:56.989310 14979 tablet_service.cc:1514] Processing DeleteTablet for tablet b22808ae58d64849882430a20e047638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:56 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59698
I20260808 00:37:56.989571 15712 tablet_replica.cc:330] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: stopping tablet replica
I20260808 00:37:56.989715 15712 raft_consensus.cc:2229] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:56.989715 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet b22808ae58d64849882430a20e047638 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:56 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:42876
I20260808 00:37:56.989903 15712 raft_consensus.cc:2258] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:56.990015 15715 tablet_replica.cc:330] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: stopping tablet replica
I20260808 00:37:56.990103 14714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:56.990289 14714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:56.990417 15715 raft_consensus.cc:2229] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:56.990422 15712 ts_tablet_manager.cc:1905] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:56.990860 15715 raft_consensus.cc:2258] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:56.991456 15715 ts_tablet_manager.cc:1905] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:56.991995 14979 tablet_service.cc:1467] Processing CreateTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 (DEFAULT_TABLE table=test-table [id=67555f05e8d242b09070684ec39ceb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.992148 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265bba80de8c47ecbc4f8fb1501b9336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.992268 15712 ts_tablet_manager.cc:1918] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:56.992396 15712 log.cc:1196] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/b22808ae58d64849882430a20e047638
I20260808 00:37:56.992700 15712 ts_tablet_manager.cc:1939] T b22808ae58d64849882430a20e047638 P c0f278b79b4f4b4bbd719fe46c264281: Deleting consensus metadata
I20260808 00:37:56.993451 15715 ts_tablet_manager.cc:1918] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:56.993539 15715 log.cc:1196] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/b22808ae58d64849882430a20e047638
I20260808 00:37:56.993616 15679 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Bootstrap starting.
I20260808 00:37:56.993803 15715 ts_tablet_manager.cc:1939] T b22808ae58d64849882430a20e047638 P 8bde4c1fa7274a968a648c642d469404: Deleting consensus metadata
I20260808 00:37:56.993862 14689 catalog_manager.cc:4841] TS c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697): tablet b22808ae58d64849882430a20e047638 (table test-table [id=dd622c723b30469fa702277430f166f7]) successfully deleted
I20260808 00:37:56.994040 15679 tablet_bootstrap.cc:654] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.994185 15487 tablet_service.cc:1467] Processing CreateTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 (DEFAULT_TABLE table=test-table [id=67555f05e8d242b09070684ec39ceb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.994290 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265bba80de8c47ecbc4f8fb1501b9336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.994295 14689 catalog_manager.cc:4841] TS 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701): tablet b22808ae58d64849882430a20e047638 (table test-table [id=dd622c723b30469fa702277430f166f7]) successfully deleted
I20260808 00:37:56.994781 15679 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: No bootstrap required, opened a new log
I20260808 00:37:56.994843 15679 ts_tablet_manager.cc:1397] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260808 00:37:56.995028 15679 raft_consensus.cc:348] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.995118 15679 raft_consensus.cc:374] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.995141 15679 raft_consensus.cc:729] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e49d816b847a09f4c5b45c6ca32fa, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.995193 15679 consensus_queue.cc:260] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.995324 15679 ts_tablet_manager.cc:1428] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:56.995381 15667 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Bootstrap starting.
I20260808 00:37:56.995733 15667 tablet_bootstrap.cc:654] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.996424 15224 tablet_service.cc:1467] Processing CreateTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 (DEFAULT_TABLE table=test-table [id=67555f05e8d242b09070684ec39ceb32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:56.996556 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 265bba80de8c47ecbc4f8fb1501b9336. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:56.997622 15667 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: No bootstrap required, opened a new log
I20260808 00:37:56.997684 15667 ts_tablet_manager.cc:1397] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:56.997853 15667 raft_consensus.cc:348] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.997936 15667 raft_consensus.cc:374] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.997956 15667 raft_consensus.cc:729] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.997960 15671 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Bootstrap starting.
I20260808 00:37:56.997998 15667 consensus_queue.cc:260] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.998081 15667 ts_tablet_manager.cc:1428] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:56.998430 15671 tablet_bootstrap.cc:654] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:56.998745 15713 tablet_replica.cc:330] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: stopping tablet replica
I20260808 00:37:56.998893 15713 raft_consensus.cc:2229] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:56.999032 15713 raft_consensus.cc:2258] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:56.999197 15671 tablet_bootstrap.cc:492] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: No bootstrap required, opened a new log
I20260808 00:37:56.999240 15671 ts_tablet_manager.cc:1397] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:56.999392 15671 raft_consensus.cc:348] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.999483 15671 raft_consensus.cc:374] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:56.999503 15671 raft_consensus.cc:729] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Initialized, Role: FOLLOWER
I20260808 00:37:56.999548 15671 consensus_queue.cc:260] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:56.999624 15671 ts_tablet_manager.cc:1428] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.000205 15713 ts_tablet_manager.cc:1905] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.002398 15713 ts_tablet_manager.cc:1918] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:57.002502 15713 log.cc:1196] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/b22808ae58d64849882430a20e047638
I20260808 00:37:57.002774 15713 ts_tablet_manager.cc:1939] T b22808ae58d64849882430a20e047638 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting consensus metadata
I20260808 00:37:57.003268 14686 catalog_manager.cc:4841] TS e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617): tablet b22808ae58d64849882430a20e047638 (table test-table [id=dd622c723b30469fa702277430f166f7]) successfully deleted
I20260808 00:37:57.049587 15672 raft_consensus.cc:482] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:57.049708 15672 raft_consensus.cc:504] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.050074 15672 leader_election.cc:288] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697)
I20260808 00:37:57.053587 15421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.050290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39617 (local address 127.1.208.3:43781)
0808 00:37:57.050354 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.050366 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.050415 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.050611 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.050612 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.050614 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:57.050673 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.050675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.051487 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.051488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.052471 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.052474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.053257 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.053258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.053370 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.053375 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.053433 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.053481 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 00:37:57.053576 15656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.050391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:43781 (local address 127.1.208.1:39617)
0808 00:37:57.050438 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.050440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.050451 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:37:57.050484 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.050485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.050494 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.050561 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:57.050751 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.051414 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.052580 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.053171 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.053250 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.053264 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.053451 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.053500 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:37:57.053865 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.050812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55445 (local address 127.1.208.2:41697)
0808 00:37:57.050884 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.050886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.050895 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:57.050927 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.050928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.050937 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.050994 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:37:57.051186 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.051905 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.052989 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.053479 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.053539 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.053551 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.053755 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.053803 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:37:57.053979 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265bba80de8c47ecbc4f8fb1501b9336" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" is_pre_election: true
I20260808 00:37:57.054085 15021 raft_consensus.cc:2454] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 0.
I20260808 00:37:57.054100 15723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.050296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41697 (local address 127.1.208.3:55445)
0808 00:37:57.050592 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.050604 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.050665 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.051016 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.051017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.051019 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:37:57.051107 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.051109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.052003 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.052005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.052905 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.052907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.053586 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.053588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.053653 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.053657 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.053973 (+ 316us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.054021 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":21,"thread_start_us":84,"threads_started":1}
I20260808 00:37:57.054276 15405 leader_election.cc:302] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.054421 15672 raft_consensus.cc:2781] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:57.054452 15672 raft_consensus.cc:482] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:57.054476 15672 raft_consensus.cc:3037] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.054404 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265bba80de8c47ecbc4f8fb1501b9336" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f278b79b4f4b4bbd719fe46c264281" is_pre_election: true
I20260808 00:37:57.054548 15267 raft_consensus.cc:2454] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 0.
I20260808 00:37:57.055267 15672 raft_consensus.cc:504] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.055406 15672 leader_election.cc:288] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 election: Requested vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697)
I20260808 00:37:57.055584 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265bba80de8c47ecbc4f8fb1501b9336" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f278b79b4f4b4bbd719fe46c264281"
I20260808 00:37:57.055681 15267 raft_consensus.cc:3037] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.055660 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "265bba80de8c47ecbc4f8fb1501b9336" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
I20260808 00:37:57.055748 15021 raft_consensus.cc:3037] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.056432 15021 raft_consensus.cc:2454] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 1.
I20260808 00:37:57.056612 15267 raft_consensus.cc:2454] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 1.
I20260808 00:37:57.056617 15405 leader_election.cc:302] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.056712 15672 raft_consensus.cc:2781] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:57.056881 15672 raft_consensus.cc:686] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 1 LEADER]: Becoming Leader. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Running, Role: LEADER
I20260808 00:37:57.056962 15672 consensus_queue.cc:237] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.057819 14714 catalog_manager.cc:5495] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bde4c1fa7274a968a648c642d469404 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "8bde4c1fa7274a968a648c642d469404" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } health_report { overall_health: HEALTHY } } }
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149476, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.62@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:57.102320 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.094236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40713 (local address 127.0.0.1:43394)
0808 00:37:57.094439 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.094451 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.094523 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.094676 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.094677 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.095000 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:37:57.095053 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.095055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.096057 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.096058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.096631 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.096633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.097076 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.097077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.097117 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.097719 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:37:57.097724 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:37:57.099391 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:37:57.101220 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:57.101232 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:57.101235 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:57.101511 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:57.101711 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:37:57.101712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:37:57.101712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:37:57.101784 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:37:57.102027 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:37:57.102033 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:37:57.102080 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.102127 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.102238 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":117,"threads_started":1}
I20260808 00:37:57.102444 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.094392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43394 (local address 127.1.208.62:40713)
0808 00:37:57.094452 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.094454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.094513 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:37:57.094576 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.094577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.094588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.094668 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:57.095152 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.095933 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.096717 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.097007 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.097068 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.099506 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:57.099508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:57.099508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:57.099520 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:57.101110 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:57.101587 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:57.101589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:57.101590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:57.101656 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:57.101830 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:57.101831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:57.101832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:57.101974 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:57.102040 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.102164 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.102310 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":33}
I20260808 00:37:57.104656 14714 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:43394:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 00:37:57.104913 14714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:57.106509 15224 tablet_service.cc:1467] Processing CreateTablet for tablet f5144b7ee42b41d4802603110ac6a8bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa307f747bbe444cabc1984933771d0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:57.106508 14979 tablet_service.cc:1467] Processing CreateTablet for tablet f5144b7ee42b41d4802603110ac6a8bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa307f747bbe444cabc1984933771d0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:57.106590 15487 tablet_service.cc:1467] Processing CreateTablet for tablet f5144b7ee42b41d4802603110ac6a8bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=fa307f747bbe444cabc1984933771d0b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 00:37:57.106696 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5144b7ee42b41d4802603110ac6a8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.106696 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5144b7ee42b41d4802603110ac6a8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.106779 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f5144b7ee42b41d4802603110ac6a8bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.108330 15671 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: Bootstrap starting.
I20260808 00:37:57.108333 15679 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: Bootstrap starting.
I20260808 00:37:57.108728 15671 tablet_bootstrap.cc:654] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.108752 15679 tablet_bootstrap.cc:654] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.109081 15667 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: Bootstrap starting.
I20260808 00:37:57.109261 15671 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: No bootstrap required, opened a new log
I20260808 00:37:57.109341 15671 ts_tablet_manager.cc:1397] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:57.109514 15667 tablet_bootstrap.cc:654] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.109472 15671 raft_consensus.cc:348] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.109551 15671 raft_consensus.cc:374] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.109570 15671 raft_consensus.cc:729] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.109601 15671 consensus_queue.cc:260] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.109694 15671 ts_tablet_manager.cc:1428] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.109797 15707 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.109877 15707 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.110090 15679 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: No bootstrap required, opened a new log
I20260808 00:37:57.110133 15679 ts_tablet_manager.cc:1397] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:57.110271 15679 raft_consensus.cc:348] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.110356 15679 raft_consensus.cc:374] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.110375 15679 raft_consensus.cc:729] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e49d816b847a09f4c5b45c6ca32fa, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.110419 15679 consensus_queue.cc:260] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.110499 15679 ts_tablet_manager.cc:1428] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:57.110656 15685 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.110716 15685 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.110847 15667 tablet_bootstrap.cc:492] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: No bootstrap required, opened a new log
I20260808 00:37:57.110906 15667 ts_tablet_manager.cc:1397] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:37:57.111063 15667 raft_consensus.cc:348] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.111150 15667 raft_consensus.cc:374] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.111171 15667 raft_consensus.cc:729] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.111241 15667 consensus_queue.cc:260] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.111348 15667 ts_tablet_manager.cc:1428] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.111438 15672 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.111507 15672 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.137112 15707 raft_consensus.cc:482] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:57.137231 15707 raft_consensus.cc:504] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.137462 15707 leader_election.cc:288] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701), e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617)
I20260808 00:37:57.137676 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5144b7ee42b41d4802603110ac6a8bb" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404" is_pre_election: true
I20260808 00:37:57.137687 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5144b7ee42b41d4802603110ac6a8bb" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" is_pre_election: true
I20260808 00:37:57.137784 15534 raft_consensus.cc:2454] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:57.137799 15021 raft_consensus.cc:2454] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:57.137943 15145 leader_election.cc:302] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, c0f278b79b4f4b4bbd719fe46c264281; no voters:
I20260808 00:37:57.138063 15707 raft_consensus.cc:2781] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:57.138119 15707 raft_consensus.cc:482] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:57.138134 15707 raft_consensus.cc:3037] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.138906 15707 raft_consensus.cc:504] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.139058 15707 leader_election.cc:288] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Requested vote from peers 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701), e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617)
I20260808 00:37:57.139243 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5144b7ee42b41d4802603110ac6a8bb" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
I20260808 00:37:57.139317 15021 raft_consensus.cc:3037] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.139278 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f5144b7ee42b41d4802603110ac6a8bb" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404"
I20260808 00:37:57.139381 15534 raft_consensus.cc:3037] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.139981 15021 raft_consensus.cc:2454] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:57.140201 15142 leader_election.cc:302] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f278b79b4f4b4bbd719fe46c264281, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.140308 15707 raft_consensus.cc:2781] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:57.140377 15707 raft_consensus.cc:686] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [term 1 LEADER]: Becoming Leader. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Running, Role: LEADER
I20260808 00:37:57.140478 15707 consensus_queue.cc:237] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } }
I20260808 00:37:57.140791 15534 raft_consensus.cc:2454] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:57.140790 15707 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: TxnStatusTablet state changed. Reason: New leader c0f278b79b4f4b4bbd719fe46c264281. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.140859 15707 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:57.141085 15743 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 00:37:57.141211 15743 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 00:37:57.141191 14714 catalog_manager.cc:5495] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:57.150774 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.147869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44836 (local address 127.1.208.2:41697)
0808 00:37:57.147925 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.147927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.147962 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:37:57.148086 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.148086 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.148099 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.148139 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.148283 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.148779 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.149998 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.150285 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.150342 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.150555 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.150590 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.150620 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.150678 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.150702 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:37:57.150812 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.147813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41697 (local address 127.0.0.1:44836)
0808 00:37:57.147915 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.147926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.147983 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.148148 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.148148 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.148150 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:57.148207 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.148210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.148850 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.148852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.149903 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.149908 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.150360 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.150362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.150437 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.150467 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.150624 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.150631 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.150668 (+ 37us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.150721 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 00:37:57.151975 15534 raft_consensus.cc:1261] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.152185 15021 raft_consensus.cc:1261] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.152288 15707 consensus_queue.cc:1035] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.152424 15688 consensus_queue.cc:1035] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.153031 15672 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: TxnStatusTablet state changed. Reason: New leader c0f278b79b4f4b4bbd719fe46c264281. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.153142 15672 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.153086 15685 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: TxnStatusTablet state changed. Reason: New leader c0f278b79b4f4b4bbd719fe46c264281. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.153178 15685 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.154088 15688 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.154220 15688 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:57.154210 15672 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.154301 15672 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P 8bde4c1fa7274a968a648c642d469404: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.154320 15698 mvcc.cc:205] Tried to move back new op lower bound from 7316068258412576768 to 7316068258367954944. Current Snapshot: MvccSnapshot[applied={T|T < 7316068258412576768 or (T in {7316068258412576768})}]
I20260808 00:37:57.155141 15684 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.155236 15684 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P c0f278b79b4f4b4bbd719fe46c264281: This TxnStatusTablet replica's current role is: LEADER
I20260808 00:37:57.155589 15685 tablet_replica.cc:439] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } } peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } }
I20260808 00:37:57.155686 15685 tablet_replica.cc:442] T f5144b7ee42b41d4802603110ac6a8bb P e48e49d816b847a09f4c5b45c6ca32fa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 00:37:57.158743 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.155897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42892 (local address 127.1.208.3:42701)
0808 00:37:57.155952 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.155954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.155964 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:37:57.156059 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.156060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.156085 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.156155 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.156269 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.156800 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.157981 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.158196 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.158252 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.158434 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.158553 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.158609 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.158654 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.158675 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:37:57.164309 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.161722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:54581 (local address 127.1.208.2:41697)
0808 00:37:57.161781 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.161783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.161792 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:37:57.161817 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.161817 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.161827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.161872 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.162033 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.162507 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.163641 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.163838 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.163892 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.164084 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.164115 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.164146 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.164215 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.164236 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:37:57.164353 15624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.161628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41697 (local address 127.1.208.3:54581)
0808 00:37:57.161697 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.161710 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.161756 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.161894 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.161895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.161896 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:57.161972 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.161974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.162578 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.162579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.163547 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.163551 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.163907 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.163908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.163978 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.164007 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.164160 (+ 153us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.164162 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.164211 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.164257 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:37:57.167098 15267 raft_consensus.cc:1261] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Refusing update from remote peer 8bde4c1fa7274a968a648c642d469404: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.167084 15021 raft_consensus.cc:1261] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Refusing update from remote peer 8bde4c1fa7274a968a648c642d469404: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.167438 15724 consensus_queue.cc:1035] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.167577 15672 consensus_queue.cc:1035] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.177354 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.173937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:59075 (local address 127.1.208.3:42701)
0808 00:37:57.174001 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.174003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.174010 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:37:57.174034 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.174035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.174046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.174088 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.174263 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.174989 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.176447 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.176748 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.176789 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.177045 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.177082 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.177108 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.177205 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.177231 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:37:57.177822 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.173734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:42701 (local address 127.1.208.2:59075)
0808 00:37:57.173912 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.173925 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.173976 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.174102 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.174103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.174104 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:57.174164 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.174166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.175201 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.175202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.176341 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.176346 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.176848 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.176849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.176919 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.176947 (+ 28us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.177147 (+ 200us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.177149 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.177676 (+ 527us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.177739 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":101,"threads_started":1}
I20260808 00:37:57.191022 14714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.191169 14714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.191948 14714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Sending DeleteTablet for 3 replicas of tablet 265bba80de8c47ecbc4f8fb1501b9336
I20260808 00:37:57.192268 14979 tablet_service.cc:1514] Processing DeleteTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59698
I20260808 00:37:57.192430 15713 tablet_replica.cc:330] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: stopping tablet replica
I20260808 00:37:57.192509 15713 raft_consensus.cc:2229] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.192557 15713 raft_consensus.cc:2258] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.192714 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 00:37:57.192809 15712 tablet_replica.cc:330] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: stopping tablet replica
I20260808 00:37:57.192858 15712 raft_consensus.cc:2229] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.192895 15712 raft_consensus.cc:2258] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.192889 15713 ts_tablet_manager.cc:1905] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.193238 15712 ts_tablet_manager.cc:1905] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.194689 15713 ts_tablet_manager.cc:1918] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:57.194756 15713 log.cc:1196] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/265bba80de8c47ecbc4f8fb1501b9336
I20260808 00:37:57.194968 15712 ts_tablet_manager.cc:1918] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:57.194991 15713 ts_tablet_manager.cc:1939] T 265bba80de8c47ecbc4f8fb1501b9336 P e48e49d816b847a09f4c5b45c6ca32fa: Deleting consensus metadata
I20260808 00:37:57.195026 15712 log.cc:1196] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/265bba80de8c47ecbc4f8fb1501b9336
I20260808 00:37:57.195225 15712 ts_tablet_manager.cc:1939] T 265bba80de8c47ecbc4f8fb1501b9336 P c0f278b79b4f4b4bbd719fe46c264281: Deleting consensus metadata
I20260808 00:37:57.195613 14686 catalog_manager.cc:4841] TS e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617): tablet 265bba80de8c47ecbc4f8fb1501b9336 (table test-table [id=67555f05e8d242b09070684ec39ceb32]) successfully deleted
I20260808 00:37:57.195919 14689 catalog_manager.cc:4841] TS c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697): tablet 265bba80de8c47ecbc4f8fb1501b9336 (table test-table [id=67555f05e8d242b09070684ec39ceb32]) successfully deleted
I20260808 00:37:57.196367 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet 265bba80de8c47ecbc4f8fb1501b9336 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:42876
I20260808 00:37:57.196491 15715 tablet_replica.cc:330] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: stopping tablet replica
I20260808 00:37:57.196552 15715 raft_consensus.cc:2229] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:57.196658 15715 raft_consensus.cc:2258] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.197049 15715 ts_tablet_manager.cc:1905] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.198812 15715 ts_tablet_manager.cc:1918] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:57.198875 15715 log.cc:1196] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/265bba80de8c47ecbc4f8fb1501b9336
I20260808 00:37:57.199082 15715 ts_tablet_manager.cc:1939] T 265bba80de8c47ecbc4f8fb1501b9336 P 8bde4c1fa7274a968a648c642d469404: Deleting consensus metadata
I20260808 00:37:57.199481 14689 catalog_manager.cc:4841] TS 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701): tablet 265bba80de8c47ecbc4f8fb1501b9336 (table test-table [id=67555f05e8d242b09070684ec39ceb32]) successfully deleted
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[14637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:37:57.235491 14815 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:37:57.227892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43408 (local address 127.1.208.62:40713)
0808 00:37:57.227955 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.227957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.228188 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:37:57.228265 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.228267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.228280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.228335 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:57.229093 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.229913 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.231076 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.231399 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.231487 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.235412 (+ 3925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43408: BlockingRecv error: recv got EOF from 127.0.0.1:43408 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[14637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM
I20260808 00:37:57.246534 14815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.236676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.1.208.62:40713)
0808 00:37:57.236741 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.236744 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.236866 (+ 122us) server_negotiation.cc:407] Connection header received
0808 00:37:57.236928 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.236930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.236942 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.237024 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:37:57.237682 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.238466 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.239530 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.239851 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.239933 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.243152 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:37:57.243155 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:37:57.243155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:37:57.243167 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:37:57.244929 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:57.245469 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:57.245470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:57.245471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:57.245546 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:37:57.245781 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:37:57.245782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:37:57.245782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:37:57.245941 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:37:57.246100 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.246207 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.246313 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:37:57.248416 14714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:57.248636 14714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:57.250350 14979 tablet_service.cc:1467] Processing CreateTablet for tablet 981b59e1551848b89556b24adeb0ff3f (DEFAULT_TABLE table=test-table [id=72a2cee803dc474fb60bcea318b72745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.250548 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b59e1551848b89556b24adeb0ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.251977 15679 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Bootstrap starting.
I20260808 00:37:57.252441 15679 tablet_bootstrap.cc:654] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.253703 15224 tablet_service.cc:1467] Processing CreateTablet for tablet 981b59e1551848b89556b24adeb0ff3f (DEFAULT_TABLE table=test-table [id=72a2cee803dc474fb60bcea318b72745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.253832 15679 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: No bootstrap required, opened a new log
I20260808 00:37:57.253856 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b59e1551848b89556b24adeb0ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.253886 15679 ts_tablet_manager.cc:1397] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:57.254062 15679 raft_consensus.cc:348] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.254138 15679 raft_consensus.cc:374] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.254158 15679 raft_consensus.cc:729] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e49d816b847a09f4c5b45c6ca32fa, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.254207 15679 consensus_queue.cc:260] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.254295 15679 ts_tablet_manager.cc:1428] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260808 00:37:57.255277 15487 tablet_service.cc:1467] Processing CreateTablet for tablet 981b59e1551848b89556b24adeb0ff3f (DEFAULT_TABLE table=test-table [id=72a2cee803dc474fb60bcea318b72745]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.255415 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 981b59e1551848b89556b24adeb0ff3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.256896 15671 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Bootstrap starting.
I20260808 00:37:57.257428 15671 tablet_bootstrap.cc:654] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.257768 15667 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Bootstrap starting.
I20260808 00:37:57.258165 15667 tablet_bootstrap.cc:654] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.258852 15667 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: No bootstrap required, opened a new log
I20260808 00:37:57.258899 15667 ts_tablet_manager.cc:1397] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:57.259039 15667 raft_consensus.cc:348] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.259119 15667 raft_consensus.cc:374] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.259137 15667 raft_consensus.cc:729] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.259176 15667 consensus_queue.cc:260] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.259258 15667 ts_tablet_manager.cc:1428] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:57.259863 15671 tablet_bootstrap.cc:492] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: No bootstrap required, opened a new log
I20260808 00:37:57.259928 15671 ts_tablet_manager.cc:1397] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260808 00:37:57.260046 15671 raft_consensus.cc:348] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.260118 15671 raft_consensus.cc:374] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.260135 15671 raft_consensus.cc:729] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.260185 15671 consensus_queue.cc:260] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.260285 15671 ts_tablet_manager.cc:1428] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.260510 15763 raft_consensus.cc:482] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:57.260572 15763 raft_consensus.cc:504] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.260699 15763 leader_election.cc:288] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697)
I20260808 00:37:57.260988 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b59e1551848b89556b24adeb0ff3f" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f278b79b4f4b4bbd719fe46c264281" is_pre_election: true
I20260808 00:37:57.261078 15267 raft_consensus.cc:2454] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 0.
I20260808 00:37:57.261482 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b59e1551848b89556b24adeb0ff3f" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" is_pre_election: true
I20260808 00:37:57.261548 15407 leader_election.cc:302] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, c0f278b79b4f4b4bbd719fe46c264281; no voters:
I20260808 00:37:57.261588 15021 raft_consensus.cc:2454] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 0.
I20260808 00:37:57.261873 15763 raft_consensus.cc:2781] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:57.261916 15763 raft_consensus.cc:482] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:57.261932 15763 raft_consensus.cc:3037] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.262771 15763 raft_consensus.cc:504] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.262912 15763 leader_election.cc:288] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 election: Requested vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697)
I20260808 00:37:57.263161 15267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b59e1551848b89556b24adeb0ff3f" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0f278b79b4f4b4bbd719fe46c264281"
I20260808 00:37:57.263170 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "981b59e1551848b89556b24adeb0ff3f" candidate_uuid: "8bde4c1fa7274a968a648c642d469404" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
I20260808 00:37:57.263250 15267 raft_consensus.cc:3037] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.263269 15021 raft_consensus.cc:3037] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.264104 15267 raft_consensus.cc:2454] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 1.
I20260808 00:37:57.264108 15021 raft_consensus.cc:2454] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8bde4c1fa7274a968a648c642d469404 in term 1.
I20260808 00:37:57.264501 15405 leader_election.cc:302] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8bde4c1fa7274a968a648c642d469404, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.264813 15763 raft_consensus.cc:2781] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:57.264879 15763 raft_consensus.cc:686] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 1 LEADER]: Becoming Leader. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Running, Role: LEADER
I20260808 00:37:57.264930 15763 consensus_queue.cc:237] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.265503 14714 catalog_manager.cc:5495] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8bde4c1fa7274a968a648c642d469404 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "8bde4c1fa7274a968a648c642d469404" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } health_report { overall_health: HEALTHY } } }
I20260808 00:37:57.273658 15665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.270572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42896 (local address 127.1.208.3:42701)
0808 00:37:57.270628 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.270630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.270666 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:37:57.270695 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.270696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.270706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.270760 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.270917 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.271611 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.272774 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.273053 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.273135 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.273363 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.273414 (+ 51us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.273471 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.273546 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.273572 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:37:57.274832 15267 raft_consensus.cc:1261] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Refusing update from remote peer 8bde4c1fa7274a968a648c642d469404: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.275002 15021 raft_consensus.cc:1261] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Refusing update from remote peer 8bde4c1fa7274a968a648c642d469404: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.275112 15672 consensus_queue.cc:1035] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.275817 15672 consensus_queue.cc:1035] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.278712 14714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.278815 14714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.279521 14714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Sending DeleteTablet for 3 replicas of tablet 981b59e1551848b89556b24adeb0ff3f
I20260808 00:37:57.279822 14979 tablet_service.cc:1514] Processing DeleteTablet for tablet 981b59e1551848b89556b24adeb0ff3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59698
I20260808 00:37:57.279915 15713 tablet_replica.cc:330] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: stopping tablet replica
I20260808 00:37:57.279963 15713 raft_consensus.cc:2229] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.279994 15713 pending_rounds.cc:62] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Trying to abort 2 pending ops.
I20260808 00:37:57.280007 15713 pending_rounds.cc:68] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7316068258915807232 op_type: WRITE_OP write_request { tablet_id: "981b59e1551848b89556b24adeb0ff3f" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\221\367\331\323\006\"8\n\ntest-admin\022*\n 72a2cee803dc474fb60bcea318b72745\020\001\030\001 \001(\001" signature: "<redacted>""\037?\322\017\343\212\262\232:\000]\2610\231\004\275\000\263\r\300Zz\3434\330\266xA\\\357#\365\2346\310\275\222\014{E\304\230XB\023\301\347\264\373\301R?\000N\000U\361\211\302\331\340\024\345\256" signing_key_seq_num: 0 } } request_id { client_id: "3e6f2164fae54b038533a7f61d2efd23" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260808 00:37:57.280200 15713 pending_rounds.cc:68] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7316068258877632512 op_type: NO_OP noop_request { }
I20260808 00:37:57.280226 15713 raft_consensus.cc:2866] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260808 00:37:57.280251 15713 raft_consensus.cc:2258] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.280802 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet 981b59e1551848b89556b24adeb0ff3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 00:37:57.280905 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet 981b59e1551848b89556b24adeb0ff3f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:42876
I20260808 00:37:57.280988 15712 tablet_replica.cc:330] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: stopping tablet replica
I20260808 00:37:57.281037 15712 raft_consensus.cc:2229] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.281078 15712 raft_consensus.cc:2258] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.281138 15715 tablet_replica.cc:330] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: stopping tablet replica
I20260808 00:37:57.281186 15715 raft_consensus.cc:2229] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:57.281242 15715 raft_consensus.cc:2258] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.281858 14714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:37:57.282049 14714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:37:57.282768 15715 ts_tablet_manager.cc:1905] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.284714 15715 ts_tablet_manager.cc:1918] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:57.284801 15715 log.cc:1196] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/981b59e1551848b89556b24adeb0ff3f
I20260808 00:37:57.285014 15715 ts_tablet_manager.cc:1939] T 981b59e1551848b89556b24adeb0ff3f P 8bde4c1fa7274a968a648c642d469404: Deleting consensus metadata
W20260808 00:37:57.285107 15795 consensus_queue.cc:1159] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" status { last_received { term: 1 index: 2 } last_committed_idx: 0 last_received_current_leader { term: 1 index: 2 } }
I20260808 00:37:57.285602 14979 tablet_service.cc:1467] Processing CreateTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef (DEFAULT_TABLE table=test-table [id=c18a934a57a843c7974ac3dba553f494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.285749 14979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5a0b359cd24e59b42916a811e9b7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.287235 15679 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: Bootstrap starting.
I20260808 00:37:57.287537 15713 ts_tablet_manager.cc:1905] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.287715 15679 tablet_bootstrap.cc:654] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.288807 15679 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: No bootstrap required, opened a new log
I20260808 00:37:57.288861 15679 ts_tablet_manager.cc:1397] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:57.288952 15713 ts_tablet_manager.cc:1918] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:57.289008 15713 log.cc:1196] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/981b59e1551848b89556b24adeb0ff3f
I20260808 00:37:57.289024 15679 raft_consensus.cc:348] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.289095 15679 raft_consensus.cc:374] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.289113 15679 raft_consensus.cc:729] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e48e49d816b847a09f4c5b45c6ca32fa, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.289158 15679 consensus_queue.cc:260] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.289216 15713 ts_tablet_manager.cc:1939] T 981b59e1551848b89556b24adeb0ff3f P e48e49d816b847a09f4c5b45c6ca32fa: Deleting consensus metadata
I20260808 00:37:57.289237 15679 ts_tablet_manager.cc:1428] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.289709 14689 catalog_manager.cc:4841] TS 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701): tablet 981b59e1551848b89556b24adeb0ff3f (table test-table [id=72a2cee803dc474fb60bcea318b72745]) successfully deleted
I20260808 00:37:57.289987 15224 tablet_service.cc:1467] Processing CreateTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef (DEFAULT_TABLE table=test-table [id=c18a934a57a843c7974ac3dba553f494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.290126 15224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5a0b359cd24e59b42916a811e9b7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.290271 15487 tablet_service.cc:1467] Processing CreateTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef (DEFAULT_TABLE table=test-table [id=c18a934a57a843c7974ac3dba553f494]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:37:57.290416 15487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad5a0b359cd24e59b42916a811e9b7ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:37:57.291402 15671 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Bootstrap starting.
I20260808 00:37:57.291702 15671 tablet_bootstrap.cc:654] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.291832 15667 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Bootstrap starting.
I20260808 00:37:57.292270 15667 tablet_bootstrap.cc:654] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Neither blocks nor log segments found. Creating new log.
I20260808 00:37:57.293113 15667 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: No bootstrap required, opened a new log
I20260808 00:37:57.293175 15667 ts_tablet_manager.cc:1397] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260808 00:37:57.293371 15667 raft_consensus.cc:348] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.293480 15667 raft_consensus.cc:374] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.293509 15667 raft_consensus.cc:729] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bde4c1fa7274a968a648c642d469404, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.293552 15667 consensus_queue.cc:260] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.293654 15667 ts_tablet_manager.cc:1428] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260808 00:37:57.293808 15671 tablet_bootstrap.cc:492] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: No bootstrap required, opened a new log
I20260808 00:37:57.293856 15671 ts_tablet_manager.cc:1397] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:37:57.293963 15671 raft_consensus.cc:348] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.294031 15671 raft_consensus.cc:374] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:37:57.294049 15671 raft_consensus.cc:729] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Initialized, Role: FOLLOWER
I20260808 00:37:57.294088 15671 consensus_queue.cc:260] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.294173 15671 ts_tablet_manager.cc:1428] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260808 00:37:57.295259 14686 catalog_manager.cc:4841] TS e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617): tablet 981b59e1551848b89556b24adeb0ff3f (table test-table [id=72a2cee803dc474fb60bcea318b72745]) successfully deleted
I20260808 00:37:57.295585 15712 ts_tablet_manager.cc:1905] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.297089 15712 ts_tablet_manager.cc:1918] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:37:57.297144 15712 log.cc:1196] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/981b59e1551848b89556b24adeb0ff3f
I20260808 00:37:57.297389 15712 ts_tablet_manager.cc:1939] T 981b59e1551848b89556b24adeb0ff3f P c0f278b79b4f4b4bbd719fe46c264281: Deleting consensus metadata
I20260808 00:37:57.297991 14689 catalog_manager.cc:4841] TS c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697): tablet 981b59e1551848b89556b24adeb0ff3f (table test-table [id=72a2cee803dc474fb60bcea318b72745]) successfully deleted
I20260808 00:37:57.430567 15688 raft_consensus.cc:482] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:37:57.430697 15688 raft_consensus.cc:504] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.430908 15688 leader_election.cc:288] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701)
I20260808 00:37:57.431190 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5a0b359cd24e59b42916a811e9b7ef" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404" is_pre_election: true
I20260808 00:37:57.431177 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5a0b359cd24e59b42916a811e9b7ef" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" is_pre_election: true
I20260808 00:37:57.431309 15021 raft_consensus.cc:2454] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:57.431310 15534 raft_consensus.cc:2454] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 0.
I20260808 00:37:57.431491 15142 leader_election.cc:302] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f278b79b4f4b4bbd719fe46c264281, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.431612 15688 raft_consensus.cc:2781] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:37:57.431670 15688 raft_consensus.cc:482] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:37:57.431686 15688 raft_consensus.cc:3037] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.432458 15688 raft_consensus.cc:504] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.432628 15688 leader_election.cc:288] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Requested vote from peers e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617), 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701)
I20260808 00:37:57.432833 15534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5a0b359cd24e59b42916a811e9b7ef" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8bde4c1fa7274a968a648c642d469404"
I20260808 00:37:57.432850 15021 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad5a0b359cd24e59b42916a811e9b7ef" candidate_uuid: "c0f278b79b4f4b4bbd719fe46c264281" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e48e49d816b847a09f4c5b45c6ca32fa"
I20260808 00:37:57.432953 15021 raft_consensus.cc:3037] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.432953 15534 raft_consensus.cc:3037] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:37:57.433725 15534 raft_consensus.cc:2454] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:57.433740 15021 raft_consensus.cc:2454] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0f278b79b4f4b4bbd719fe46c264281 in term 1.
I20260808 00:37:57.433938 15142 leader_election.cc:302] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c0f278b79b4f4b4bbd719fe46c264281, e48e49d816b847a09f4c5b45c6ca32fa; no voters:
I20260808 00:37:57.434053 15688 raft_consensus.cc:2781] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:37:57.434109 15688 raft_consensus.cc:686] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 1 LEADER]: Becoming Leader. State: Replica: c0f278b79b4f4b4bbd719fe46c264281, State: Running, Role: LEADER
I20260808 00:37:57.434155 15688 consensus_queue.cc:237] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } }
I20260808 00:37:57.434855 14715 catalog_manager.cc:5495] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "c0f278b79b4f4b4bbd719fe46c264281" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0f278b79b4f4b4bbd719fe46c264281" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 } health_report { overall_health: UNKNOWN } } }
I20260808 00:37:57.446391 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.444061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.1.208.2:41697)
0808 00:37:57.444124 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.444126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.444134 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:37:57.444155 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.444156 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.444166 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.444212 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.444388 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.444945 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.445767 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.445977 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.446032 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.446170 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.446200 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.446231 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.446295 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.446311 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:37:57.450925 15762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.448605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41697 (local address 127.1.208.2:60733)
0808 00:37:57.448659 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.448670 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:37:57.448703 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:37:57.448882 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:37:57.448883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:37:57.448884 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 00:37:57.448927 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.448928 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.449620 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.449621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.450195 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:37:57.450197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.450564 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.450566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:37:57.450613 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.450619 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.450748 (+ 129us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.450749 (+ 1us) client_negotiation.cc:769] Sending connection context
0808 00:37:57.450796 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:37:57.450842 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 00:37:57.450925 15657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.448754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:60733 (local address 127.1.208.2:41697)
0808 00:37:57.448798 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:37:57.448800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:37:57.448806 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:37:57.448825 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:37:57.448825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:37:57.448833 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:37:57.448874 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:37:57.448972 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.449550 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.450273 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:37:57.450500 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:37:57.450559 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:37:57.450673 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:37:57.450707 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:37:57.450749 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:37:57.450799 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 00:37:57.450817 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:37:57.453157 15534 raft_consensus.cc:1261] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.453154 15021 raft_consensus.cc:1261] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Refusing update from remote peer c0f278b79b4f4b4bbd719fe46c264281: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:37:57.453397 15688 consensus_queue.cc:1035] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e48e49d816b847a09f4c5b45c6ca32fa" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.453536 15684 consensus_queue.cc:1035] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8bde4c1fa7274a968a648c642d469404" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 42701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:37:57.464134 14714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.464258 14714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:37:57.464970 14714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dd96e5fd1fc4ea7a698335a34096577: Sending DeleteTablet for 3 replicas of tablet ad5a0b359cd24e59b42916a811e9b7ef
I20260808 00:37:57.465230 15224 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:44818
I20260808 00:37:57.465364 15712 tablet_replica.cc:330] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: stopping tablet replica
I20260808 00:37:57.465339 14979 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59698
I20260808 00:37:57.465443 15713 tablet_replica.cc:330] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa: stopping tablet replica
I20260808 00:37:57.465459 15712 raft_consensus.cc:2229] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:37:57.465492 15713 raft_consensus.cc:2229] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.465528 15713 raft_consensus.cc:2258] T ad5a0b359cd24e59b42916a811e9b7ef P e48e49d816b847a09f4c5b45c6ca32fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.465538 15712 raft_consensus.cc:2258] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.465809 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 14859
I20260808 00:37:57.465852 15712 ts_tablet_manager.cc:1905] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.480340 15712 ts_tablet_manager.cc:1918] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:57.480417 15712 log.cc:1196] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/ad5a0b359cd24e59b42916a811e9b7ef
I20260808 00:37:57.480584 15712 ts_tablet_manager.cc:1939] T ad5a0b359cd24e59b42916a811e9b7ef P c0f278b79b4f4b4bbd719fe46c264281: Deleting consensus metadata
I20260808 00:37:57.481063 14689 catalog_manager.cc:4841] TS c0f278b79b4f4b4bbd719fe46c264281 (127.1.208.2:41697): tablet ad5a0b359cd24e59b42916a811e9b7ef (table test-table [id=c18a934a57a843c7974ac3dba553f494]) successfully deleted
I20260808 00:37:57.482743 15487 tablet_service.cc:1514] Processing DeleteTablet for tablet ad5a0b359cd24e59b42916a811e9b7ef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:37:57 UTC) from {username='oryx', principal='oryx/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:42876
I20260808 00:37:57.482899 15715 tablet_replica.cc:330] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: stopping tablet replica
I20260808 00:37:57.482964 15715 raft_consensus.cc:2229] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:37:57.483016 15715 raft_consensus.cc:2258] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:37:57.483394 15715 ts_tablet_manager.cc:1905] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:37:57.485059 15715 ts_tablet_manager.cc:1918] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 00:37:57.485124 15715 log.cc:1196] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/ad5a0b359cd24e59b42916a811e9b7ef
I20260808 00:37:57.485442 15715 ts_tablet_manager.cc:1939] T ad5a0b359cd24e59b42916a811e9b7ef P 8bde4c1fa7274a968a648c642d469404: Deleting consensus metadata
I20260808 00:37:57.485841 14689 catalog_manager.cc:4841] TS 8bde4c1fa7274a968a648c642d469404 (127.1.208.3:42701): tablet ad5a0b359cd24e59b42916a811e9b7ef (table test-table [id=c18a934a57a843c7974ac3dba553f494]) successfully deleted
W20260808 00:37:57.487682 14686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:39617 (error 108)
I20260808 00:37:57.487975 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 15110
I20260808 00:37:57.503774 15817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:37:57.488038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39617 (local address 127.0.0.1:59712)
0808 00:37:57.503633 (+ 15595us) negotiation.cc:107] Waiting for socket to connect
0808 00:37:57.503720 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:39617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15568,"thread_start_us":45,"threads_started":1}
W20260808 00:37:57.503947 14686 catalog_manager.cc:4584] TS e48e49d816b847a09f4c5b45c6ca32fa (127.1.208.1:39617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad5a0b359cd24e59b42916a811e9b7ef: Network error: Client connection negotiation failed: client connection to 127.1.208.1:39617: connect: Connection refused (error 111)
I20260808 00:37:57.511708 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 15372
I20260808 00:37:57.527734 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 14655
2026-08-08T00:37:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2812 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[15823](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[15823](info): set up 2 sockets
Aug 08 00:37:57 dist-test-slave-gjw7 krb5kdc[15823](info): commencing operation
krb5kdc: starting...
W20260808 00:37:58.750335 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:37:58 dist-test-slave-gjw7 krb5kdc[15823](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:37:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:37:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:37:58.871678 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:36945
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:36945
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:37:58.953020 15848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:37:58.953224 15848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:37:58.953253 15848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:37:58.954425 15848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:37:58.954466 15848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:37:58.954486 15848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:37:58.954501 15848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:37:58.954517 15848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:37:58.956024 15848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:36945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:36945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:37:58.956264 15848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:37:58.956471 15848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:37:58.961125 15862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:37:58.961112 15857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:58.961283 15848 server_base.cc:1034] running on GCE node
W20260808 00:37:58.961140 15855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:37:58.961638 15848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:37:58.961889 15848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:37:58.963030 15848 hybrid_clock.cc:648] HybridClock initialized: now 1786149478963018 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T00:37:58Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1430 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:37:59 dist-test-slave-gjw7 krb5kdc[15871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:37:59 dist-test-slave-gjw7 krb5kdc[15871](info): set up 2 sockets
Aug 08 00:37:59 dist-test-slave-gjw7 krb5kdc[15871](info): commencing operation
krb5kdc: starting...
W20260808 00:38:00.134845 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:38:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:38:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:38:00.250372 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37481
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:00.330847 15892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:00.331071 15892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:00.331101 15892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:00.332247 15892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:38:00.332300 15892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:00.332319 15892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:00.332335 15892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:38:00.332350 15892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:38:00.333856 15892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:00.334106 15892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:00.334304 15892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:00.339149 15905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:00.339177 15892 server_base.cc:1034] running on GCE node
W20260808 00:38:00.339150 15903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.339177 15907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:00.339670 15892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:00.339936 15892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:00.341071 15892 hybrid_clock.cc:648] HybridClock initialized: now 1786149480341053 us; error 34 us; skew 500 ppm
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:00.343480 15892 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:38:00.343854 15892 webserver.cc:460] Webserver started at http://127.1.208.62:42423/ using document root <none> and password file <none>
I20260808 00:38:00.344024 15892 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:00.344061 15892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:00.344151 15892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:00.345166 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "b55c006616a840048c3160254630f215"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "6f9438c88e086740517d1d36987b38b5"
server_key_iv: "f30668e36895cd2d287d2b5566e8dacb"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.345513 15892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "b55c006616a840048c3160254630f215"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "6f9438c88e086740517d1d36987b38b5"
server_key_iv: "f30668e36895cd2d287d2b5566e8dacb"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.346786 15892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:38:00.347405 15921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.347573 15892 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.347654 15892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "b55c006616a840048c3160254630f215"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "6f9438c88e086740517d1d36987b38b5"
server_key_iv: "f30668e36895cd2d287d2b5566e8dacb"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.347707 15892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:00.358314 15892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:00.359905 15892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:00.360005 15892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:00.363878 15892 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37481
I20260808 00:38:00.363929 16021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37481 every 8 connection(s)
I20260808 00:38:00.364228 15892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:38:00.364806 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 15892
I20260808 00:38:00.364910 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:38:00.365087 1856 external_mini_cluster.cc:1428] Setting key 45be12e2a4224d6a7b57371cb251129f
I20260808 00:38:00.365099 16024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:00.368373 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Bootstrap starting.
I20260808 00:38:00.368975 16024 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:00.369256 16024 log.cc:824] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:00.370014 16024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: No bootstrap required, opened a new log
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:00.372061 16024 raft_consensus.cc:348] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:00.372227 16024 raft_consensus.cc:374] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:00.372256 16024 raft_consensus.cc:729] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b55c006616a840048c3160254630f215, State: Initialized, Role: FOLLOWER
I20260808 00:38:00.372398 16024 consensus_queue.cc:260] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:00.372468 16024 raft_consensus.cc:388] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:38:00.372498 16024 raft_consensus.cc:482] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:38:00.372545 16024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:00.373004 16024 raft_consensus.cc:504] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:00.373098 16024 leader_election.cc:302] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b55c006616a840048c3160254630f215; no voters:
I20260808 00:38:00.373270 16024 leader_election.cc:288] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:38:00.373349 16032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:38:00.373490 16032 raft_consensus.cc:686] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 LEADER]: Becoming Leader. State: Replica: b55c006616a840048c3160254630f215, State: Running, Role: LEADER
I20260808 00:38:00.373562 16024 sys_catalog.cc:564] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:38:00.373617 16032 consensus_queue.cc:237] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:00.373943 16032 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b55c006616a840048c3160254630f215. Latest consensus state: current_term: 1 leader_uuid: "b55c006616a840048c3160254630f215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } } }
I20260808 00:38:00.374022 16032 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:00.374185 16038 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b55c006616a840048c3160254630f215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } } }
I20260808 00:38:00.374259 16038 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: This master's current role is: LEADER
W20260808 00:38:00.375448 16061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:38:00.375502 16061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:38:00.375617 16046 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:38:00.375810 16046 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:38:00.375890 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.365569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.1.208.62:37481)
0808 00:38:00.365823 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:00.365832 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:00.365868 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:38:00.366284 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:00.366292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:00.366366 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:00.366509 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:00.367081 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.368383 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.369160 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.369704 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.369810 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.372190 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:00.372194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:00.372194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:00.372205 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:00.374091 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.374515 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.374518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.374518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.374625 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.374856 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.374857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.374857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.375179 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:00.375342 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:00.375455 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 00:38:00.375684 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":73,"threads_started":1}
I20260808 00:38:00.377383 16046 catalog_manager.cc:1347] Generated new cluster ID: 884c678a0b394e2da9bceb5781a0b3a6
I20260808 00:38:00.377437 16046 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:38:00.395696 16046 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:38:00.396229 16046 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:38:00.414458 16046 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Generated new TSK 0
I20260808 00:38:00.414662 16046 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:38:00.447734 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:00.529482 16071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:00.529685 16071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:00.529716 16071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:00.530871 16071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:00.530913 16071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:00.530949 16071 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:38:00.532392 16071 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:00.532630 16071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:00.532845 16071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:00.537693 16088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.537708 16090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.537768 16093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:00.537894 16071 server_base.cc:1034] running on GCE node
I20260808 00:38:00.538132 16071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:00.538375 16071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:00.539515 16071 hybrid_clock.cc:648] HybridClock initialized: now 1786149480539498 us; error 34 us; skew 500 ppm
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:00.541978 16071 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:38:00.542387 16071 webserver.cc:460] Webserver started at http://127.1.208.1:37231/ using document root <none> and password file <none>
I20260808 00:38:00.542603 16071 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:00.542651 16071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:00.542814 16071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:00.543855 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "4fa48654cbfb40aaa54300fe93c2fe7a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a68bdc08352538737ebb315c88c3a938"
server_key_iv: "2fb96fc6fd33a950d878b07c36937b3e"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.544188 16071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "4fa48654cbfb40aaa54300fe93c2fe7a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a68bdc08352538737ebb315c88c3a938"
server_key_iv: "2fb96fc6fd33a950d878b07c36937b3e"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.545435 16071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:38:00.546125 16107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.546281 16071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:00.546363 16071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "4fa48654cbfb40aaa54300fe93c2fe7a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a68bdc08352538737ebb315c88c3a938"
server_key_iv: "2fb96fc6fd33a950d878b07c36937b3e"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.546424 16071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:00.558569 16071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:00.560305 16071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:00.560413 16071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:00.560595 16071 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:00.560848 16071 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:00.560878 16071 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.560897 16071 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:00.560913 16071 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.566391 16071 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39465
I20260808 00:38:00.566454 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39465 every 8 connection(s)
I20260808 00:38:00.566730 16071 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:00.573207 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 16071
I20260808 00:38:00.573323 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:38:00.573516 1856 external_mini_cluster.cc:1428] Setting key 8ca1f6221f0f125954911b76a2e98312
I20260808 00:38:00.576718 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.567436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:46791 (local address 127.1.208.62:37481)
0808 00:38:00.567483 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:00.567486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:00.567655 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:38:00.568187 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:00.568188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:00.568202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:00.568255 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:00.569194 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.569745 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.570718 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.570924 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.570980 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.572946 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:00.572947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:00.572947 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:00.572955 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:00.574645 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.575103 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.575105 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.575105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.575197 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.575970 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.575971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.575971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.576125 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:00.576185 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:00.576533 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 00:38:00.576621 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:38:00.577189 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.567383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:46791)
0808 00:38:00.567532 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:00.567575 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:00.568082 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:00.568341 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:00.568345 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:00.568902 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:00.569091 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.569096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.569846 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.569847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.570634 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.570636 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.570993 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.570994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.571119 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.571565 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:00.571573 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:00.572857 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:00.574751 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.574760 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.574763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.575042 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.575731 (+ 689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.575731 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.575732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.575881 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.576187 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:00.576195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:00.576369 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 00:38:00.576808 (+ 439us) client_negotiation.cc:240] Negotiation successful
0808 00:38:00.576987 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260808 00:38:00.577672 16315 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37481
I20260808 00:38:00.577785 16315 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:00.577960 16315 heartbeater.cc:511] Master 127.1.208.62:37481 requested a full tablet report, sending...
I20260808 00:38:00.578528 15957 ts_manager.cc:194] Registered new tserver with Master: 4fa48654cbfb40aaa54300fe93c2fe7a (127.1.208.1:39465)
I20260808 00:38:00.579124 15957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:46791
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:38:00.599248 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:00.680938 16321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:00.681140 16321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:00.681169 16321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:00.682410 16321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:00.682454 16321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:00.682489 16321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:38:00.683895 16321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:00.684139 16321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:00.684338 16321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:00.689217 16333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.689222 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.689333 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:00.689347 16321 server_base.cc:1034] running on GCE node
I20260808 00:38:00.689675 16321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:00.689888 16321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:00.691025 16321 hybrid_clock.cc:648] HybridClock initialized: now 1786149480691012 us; error 25 us; skew 500 ppm
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:00.693435 16321 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:38:00.693881 16321 webserver.cc:460] Webserver started at http://127.1.208.2:42307/ using document root <none> and password file <none>
I20260808 00:38:00.694103 16321 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:00.694145 16321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:00.694271 16321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:00.695282 16321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "d94d908c4d264aa2ad149b38be884179"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "16edd1a3e096ef2d7a88723504b4d563"
server_key_iv: "bb25b561ef10a2f38ab38446fd31b88f"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.695624 16321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "d94d908c4d264aa2ad149b38be884179"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "16edd1a3e096ef2d7a88723504b4d563"
server_key_iv: "bb25b561ef10a2f38ab38446fd31b88f"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.696848 16321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:38:00.697548 16345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.697691 16321 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:00.697778 16321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "d94d908c4d264aa2ad149b38be884179"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "16edd1a3e096ef2d7a88723504b4d563"
server_key_iv: "bb25b561ef10a2f38ab38446fd31b88f"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.697839 16321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:00.721154 16321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:00.722934 16321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:00.723044 16321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:00.723233 16321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:00.723492 16321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:00.723521 16321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.723543 16321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:00.723558 16321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.729400 16321 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:40333
I20260808 00:38:00.729487 16546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:40333 every 8 connection(s)
I20260808 00:38:00.729765 16321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:38:00.734105 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 16321
I20260808 00:38:00.734220 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:38:00.734436 1856 external_mini_cluster.cc:1428] Setting key 3cc7fb89cabcc50750a2581f2e9eff49
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:00.738683 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.730447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:57471 (local address 127.1.208.62:37481)
0808 00:38:00.730509 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:00.730511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:00.730683 (+ 172us) server_negotiation.cc:407] Connection header received
0808 00:38:00.731164 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:00.731165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:00.731175 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:00.731222 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:00.732122 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.732619 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.733598 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.733778 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.733826 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.736011 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:00.736013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:00.736013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:00.736021 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:00.737393 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.737832 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.737833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.737833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.737880 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.738120 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.738120 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.738121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.738213 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:00.738255 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:00.738554 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 00:38:00.738619 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:38:00.738915 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.730414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:57471)
0808 00:38:00.730567 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:00.730603 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:00.731088 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:00.731279 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:00.731282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:00.731841 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:00.732055 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.732061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.732683 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.732684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.733534 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.733536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.733832 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.733832 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.733954 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.734487 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:00.734498 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:00.735914 (+ 1416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:00.737467 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.737473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.737476 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.737762 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.737930 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.737931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.737931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.738062 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.738267 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:00.738273 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:00.738431 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 00:38:00.738545 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 00:38:00.738702 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260808 00:38:00.739419 16548 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37481
I20260808 00:38:00.739517 16548 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:00.739691 16548 heartbeater.cc:511] Master 127.1.208.62:37481 requested a full tablet report, sending...
I20260808 00:38:00.740082 15957 ts_manager.cc:194] Registered new tserver with Master: d94d908c4d264aa2ad149b38be884179 (127.1.208.2:40333)
I20260808 00:38:00.740594 15957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:57471
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:38:00.760527 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:00.842360 16557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:00.842572 16557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:00.842594 16557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:00.843715 16557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:00.843751 16557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:00.843775 16557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:38:00.845276 16557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:00.845515 16557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:00.845702 16557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:00.850665 16563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.850667 16567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:00.850685 16572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:00.850785 16557 server_base.cc:1034] running on GCE node
I20260808 00:38:00.851138 16557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:00.851341 16557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:00.852473 16557 hybrid_clock.cc:648] HybridClock initialized: now 1786149480852454 us; error 34 us; skew 500 ppm
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:00.854867 16557 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:38:00.855341 16557 webserver.cc:460] Webserver started at http://127.1.208.3:46181/ using document root <none> and password file <none>
I20260808 00:38:00.855549 16557 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:00.855595 16557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:00.855721 16557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:00.856770 16557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "b1dd51c9fec54a32a5f1504e3db2916a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a0dd10253b497adcba753431fe271ee0"
server_key_iv: "c3f23e0b23f675914b11a9be16acca48"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.857116 16557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "b1dd51c9fec54a32a5f1504e3db2916a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a0dd10253b497adcba753431fe271ee0"
server_key_iv: "c3f23e0b23f675914b11a9be16acca48"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.858530 16557 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:38:00.859189 16588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.859340 16557 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:00.859424 16557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "b1dd51c9fec54a32a5f1504e3db2916a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a0dd10253b497adcba753431fe271ee0"
server_key_iv: "c3f23e0b23f675914b11a9be16acca48"
server_key_version: "encryptionkey@0"
I20260808 00:38:00.859493 16557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:00.891578 16557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:00.893383 16557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:00.893519 16557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:00.893749 16557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:00.894062 16557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:00.894114 16557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.894155 16557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:00.894179 16557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:00.899834 16557 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:35877
I20260808 00:38:00.899892 16759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:35877 every 8 connection(s)
I20260808 00:38:00.900189 16557 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:38:00.905853 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 16557
Aug 08 00:38:00 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:00.905968 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:38:00.906134 1856 external_mini_cluster.cc:1428] Setting key 8af73a0f116350f6905f1e1bd40d34ca
I20260808 00:38:00.908752 16028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.900887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:50071 (local address 127.1.208.62:37481)
0808 00:38:00.900946 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:00.900949 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:00.901155 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:38:00.901691 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:00.901692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:00.901704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:00.901752 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:00.902669 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.903159 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.904062 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.904245 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.904290 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.906448 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:00.906449 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:00.906449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:00.906457 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:00.907703 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.908063 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.908064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.908064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.908111 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:00.908275 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:00.908276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:00.908276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:00.908372 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:00.908417 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:00.908628 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 00:38:00.908687 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:38:00.908944 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:00.900873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:50071)
0808 00:38:00.901040 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:00.901076 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:00.901605 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:00.901812 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:00.901815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:00.902384 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:00.902588 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.902594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.903221 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.903222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.904004 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:00.904006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:00.904301 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:00.904301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:00.904397 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:00.904856 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:00.904865 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:00.906363 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:00.907770 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.907777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.907778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.908006 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.908151 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:00.908151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:00.908151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:00.908238 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:00.908417 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:00.908420 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:00.908533 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:38:00.908612 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:38:00.908732 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":87,"threads_started":1}
I20260808 00:38:00.909403 16766 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37481
I20260808 00:38:00.909497 16766 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:00.909672 16766 heartbeater.cc:511] Master 127.1.208.62:37481 requested a full tablet report, sending...
I20260808 00:38:00.910030 15957 ts_manager.cc:194] Registered new tserver with Master: b1dd51c9fec54a32a5f1504e3db2916a (127.1.208.3:35877)
I20260808 00:38:00.910410 15957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:50071
I20260808 00:38:00.917744 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
Entry for principal oryx/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab.
WARNING: no policy specified for oryx/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
Entry for principal oryx/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab.
WARNING: no policy specified for oryx/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
Entry for principal oryx/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab.
I20260808 00:38:00.995949 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 16071
I20260808 00:38:01.020547 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 16321
I20260808 00:38:01.030514 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 16557
I20260808 00:38:01.046288 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 15892
I20260808 00:38:01.056232 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37481
--webserver_interface=127.1.208.62
--webserver_port=42423
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37481
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:01.139209 16785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:01.139410 16785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:01.139431 16785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:01.140573 16785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:38:01.140614 16785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:01.140626 16785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:01.140648 16785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:38:01.140671 16785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:38:01.142189 16785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37481
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=42423
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:01.142438 16785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:01.142653 16785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:01.147220 16795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.147235 16799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.147432 16785 server_base.cc:1034] running on GCE node
W20260808 00:38:01.147531 16796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.147789 16785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:01.148057 16785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:01.149196 16785 hybrid_clock.cc:648] HybridClock initialized: now 1786149481149183 us; error 28 us; skew 500 ppm
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149481, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:01.151696 16785 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:38:01.152094 16785 webserver.cc:460] Webserver started at http://127.1.208.62:42423/ using document root <none> and password file <none>
I20260808 00:38:01.152292 16785 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:01.152340 16785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:01.153630 16785 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:38:01.154211 16809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.154369 16785 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:38:01.154452 16785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "b55c006616a840048c3160254630f215"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "6f9438c88e086740517d1d36987b38b5"
server_key_iv: "f30668e36895cd2d287d2b5566e8dacb"
server_key_version: "encryptionkey@0"
I20260808 00:38:01.154783 16785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:01.178627 16785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:01.180444 16785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:01.180557 16785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:01.184491 16785 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37481
I20260808 00:38:01.184538 16909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37481 every 8 connection(s)
I20260808 00:38:01.184893 16785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:38:01.186419 16910 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:38:01.187095 16910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Bootstrap starting.
I20260808 00:38:01.191537 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 16785
I20260808 00:38:01.191890 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:39465
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:37481
--webserver_port=37231
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:38:01.196272 16910 log.cc:824] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:01.197929 16910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:38:01.198174 16910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Bootstrap complete.
I20260808 00:38:01.200284 16910 raft_consensus.cc:348] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:01.200513 16910 raft_consensus.cc:729] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b55c006616a840048c3160254630f215, State: Initialized, Role: FOLLOWER
I20260808 00:38:01.200664 16910 consensus_queue.cc:260] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:01.200739 16910 raft_consensus.cc:388] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:38:01.200790 16910 raft_consensus.cc:482] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:38:01.200836 16910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:38:01.201882 16910 raft_consensus.cc:504] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:01.201988 16910 leader_election.cc:302] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b55c006616a840048c3160254630f215; no voters:
I20260808 00:38:01.202147 16910 leader_election.cc:288] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:38:01.202193 16919 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:38:01.202317 16919 raft_consensus.cc:686] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [term 2 LEADER]: Becoming Leader. State: Replica: b55c006616a840048c3160254630f215, State: Running, Role: LEADER
I20260808 00:38:01.202416 16919 consensus_queue.cc:237] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } }
I20260808 00:38:01.202421 16910 sys_catalog.cc:564] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:38:01.203157 16921 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b55c006616a840048c3160254630f215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } } }
I20260808 00:38:01.203249 16921 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:01.203146 16922 sys_catalog.cc:455] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b55c006616a840048c3160254630f215. Latest consensus state: current_term: 2 leader_uuid: "b55c006616a840048c3160254630f215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b55c006616a840048c3160254630f215" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37481 } } }
I20260808 00:38:01.203434 16922 sys_catalog.cc:458] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:01.203538 16932 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:38:01.204087 16932 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:38:01.204213 16932 catalog_manager.cc:1259] Loaded cluster ID: 884c678a0b394e2da9bceb5781a0b3a6
I20260808 00:38:01.204244 16932 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:38:01.205034 16932 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:38:01.205199 16932 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b55c006616a840048c3160254630f215: Loaded TSK: 0
I20260808 00:38:01.205446 16932 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:38:01.297386 16912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:01.297614 16912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:01.297646 16912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:01.298816 16912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:01.298859 16912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:01.298894 16912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:38:01.300336 16912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.1.keytab
--principal=oryx/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:39465
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=37231
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:01.300567 16912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:01.300779 16912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:01.305778 16948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.305819 16954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.306034 16912 server_base.cc:1034] running on GCE node
W20260808 00:38:01.306173 16949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.306407 16912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:01.306712 16912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:01.307900 16912 hybrid_clock.cc:648] HybridClock initialized: now 1786149481307883 us; error 29 us; skew 500 ppm
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149481, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:01.310971 16912 init.cc:377] Logged in from keytab as oryx/127.1.208.1@KRBTEST.COM (short username oryx)
I20260808 00:38:01.311453 16912 webserver.cc:460] Webserver started at http://127.1.208.1:37231/ using document root <none> and password file <none>
I20260808 00:38:01.311673 16912 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:01.311730 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:01.313158 16912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:38:01.313779 16964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.313931 16912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.314013 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "4fa48654cbfb40aaa54300fe93c2fe7a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a68bdc08352538737ebb315c88c3a938"
server_key_iv: "2fb96fc6fd33a950d878b07c36937b3e"
server_key_version: "encryptionkey@0"
I20260808 00:38:01.314378 16912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:01.324824 16912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:01.326956 16912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:01.327070 16912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:01.327299 16912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:01.327620 16912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:01.327656 16912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.327687 16912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:01.327710 16912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.334352 16912 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39465
I20260808 00:38:01.334438 17183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39465 every 8 connection(s)
I20260808 00:38:01.334748 16912 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:38:01.338023 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 16912
I20260808 00:38:01.338297 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:40333
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:37481
--webserver_port=42307
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:38:01.343940 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.335592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:50407 (local address 127.1.208.62:37481)
0808 00:38:01.335800 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.335809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.335846 (+ 37us) server_negotiation.cc:407] Connection header received
0808 00:38:01.336412 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.336421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.336480 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.336635 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.337605 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.338973 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.340261 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.340693 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.340800 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.343727 (+ 2927us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:50407: BlockingRecv error: recv got EOF from 127.1.208.1:50407 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":93,"threads_started":1}
I20260808 00:38:01.343958 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.335499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:50407)
0808 00:38:01.335661 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.335703 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.336192 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.336659 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.336662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.337231 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.337496 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.337503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.339085 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.339087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.340177 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.340181 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.340780 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.340782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.340922 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.341418 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.341427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.343558 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
W20260808 00:38:01.344188 17184 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37481 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.208.62:37481: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.348505 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.344382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:47817)
0808 00:38:01.344517 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.344528 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.344578 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.344701 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.344701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.344860 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.344912 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.344913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.345564 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.345565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.346084 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.346086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.346416 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.346418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.346470 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.346855 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.346859 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.348423 (+ 1564us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
W20260808 00:38:01.348600 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.344563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:47817 (local address 127.1.208.62:37481)
0808 00:38:01.344595 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.344597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.344604 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:38:01.344625 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.344626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.344637 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.344688 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.344985 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.345489 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.346155 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.346338 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.346391 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.348538 (+ 2147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:47817: BlockingRecv error: recv got EOF from 127.1.208.1:47817 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.352726 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.348948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:60703)
0808 00:38:01.348988 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.348997 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.349031 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.349140 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.349141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.349257 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.349328 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.349330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.349903 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.349904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.350389 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.350390 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.350697 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.350699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.350768 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.351132 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.351136 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.352657 (+ 1521us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 00:38:01.352916 17184 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:38:01.353044 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.349006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60703 (local address 127.1.208.62:37481)
0808 00:38:01.349038 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.349039 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.349048 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:01.349079 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.349080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.349089 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.349131 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.349394 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.349839 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.350464 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.350641 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.350693 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.352743 (+ 2050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:60703: BlockingRecv error: recv got EOF from 127.1.208.1:60703 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 00:38:01.429701 17189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:01.429949 17189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:01.429976 17189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:01.431170 17189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:01.431216 17189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:01.431252 17189 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:38:01.432729 17189 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.2.keytab
--principal=oryx/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:40333
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=42307
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:01.432981 17189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:01.433202 17189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:01.438107 17195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.438256 17201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.438405 17196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.438284 17189 server_base.cc:1034] running on GCE node
I20260808 00:38:01.438606 17189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:01.438812 17189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:01.439960 17189 hybrid_clock.cc:648] HybridClock initialized: now 1786149481439951 us; error 31 us; skew 500 ppm
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149481, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:01.442469 17189 init.cc:377] Logged in from keytab as oryx/127.1.208.2@KRBTEST.COM (short username oryx)
I20260808 00:38:01.442878 17189 webserver.cc:460] Webserver started at http://127.1.208.2:42307/ using document root <none> and password file <none>
I20260808 00:38:01.443068 17189 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:01.443120 17189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:01.444434 17189 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:38:01.444977 17211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.445132 17189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:01.445214 17189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "d94d908c4d264aa2ad149b38be884179"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "16edd1a3e096ef2d7a88723504b4d563"
server_key_iv: "bb25b561ef10a2f38ab38446fd31b88f"
server_key_version: "encryptionkey@0"
I20260808 00:38:01.445631 17189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:01.457669 17189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:01.459616 17189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:01.459753 17189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:01.459998 17189 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:01.460325 17189 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:01.460369 17189 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.460392 17189 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:01.460415 17189 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.466167 17189 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:40333
I20260808 00:38:01.466243 17407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:40333 every 8 connection(s)
I20260808 00:38:01.466568 17189 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.473886 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.467288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:44197)
0808 00:38:01.467467 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.467506 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.468028 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.468252 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.468256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.468866 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.469080 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.469086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.469783 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.469784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.470598 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.470601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.470975 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.470975 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.471080 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.471587 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.471596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.473728 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260808 00:38:01.473899 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.467299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:44197 (local address 127.1.208.62:37481)
0808 00:38:01.467381 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.467384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.467591 (+ 207us) server_negotiation.cc:407] Connection header received
0808 00:38:01.468123 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.468125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.468139 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.468191 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.469160 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.469705 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.470700 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.470902 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.470964 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.473846 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:44197: BlockingRecv error: recv got EOF from 127.1.208.2:44197 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 00:38:01.474159 17410 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37481 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.208.62:37481: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
I20260808 00:38:01.474924 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 17189
I20260808 00:38:01.475225 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:35877
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:37481
--webserver_port=46181
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.478895 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.474387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:56105)
0808 00:38:01.474459 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.474473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.474521 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.474657 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.474658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.474779 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.474827 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.474829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.475614 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.475615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.476133 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.476135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.476555 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.476556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.476614 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.477050 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.477054 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.478829 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 00:38:01.479084 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.474475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:56105 (local address 127.1.208.62:37481)
0808 00:38:01.474524 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.474526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.474556 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:38:01.474582 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.474583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.474591 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.474635 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.474893 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.475543 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.476211 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.476478 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.476548 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.478936 (+ 2388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:56105: BlockingRecv error: recv got EOF from 127.1.208.2:56105 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.483543 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.479162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:39001)
0808 00:38:01.479249 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.479258 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.479292 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.479448 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.479449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.479582 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.479614 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.479616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.480217 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.480218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.480956 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.480959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.481322 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.481323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.481376 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.481747 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.481752 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.483488 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 00:38:01.483680 17410 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:38:01.483726 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.479301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:39001 (local address 127.1.208.62:37481)
0808 00:38:01.479345 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.479347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.479356 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:01.479376 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.479377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.479384 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.479433 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.479676 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.480143 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.481054 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.481233 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.481308 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.483662 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:39001: BlockingRecv error: recv got EOF from 127.1.208.2:39001 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:38:01.561616 17416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:01.561821 17416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:01.561844 17416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:01.562973 17416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:01.563014 17416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:01.563038 17416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:38:01.564467 17416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/krb5kdc/oryx_127.1.208.3.keytab
--principal=oryx/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:35877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=46181
--tserver_master_addrs=127.1.208.62:37481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:01.564666 17416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:01.564858 17416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:01.569736 17428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.569803 17425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:01.569764 17426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:01.569890 17416 server_base.cc:1034] running on GCE node
I20260808 00:38:01.570115 17416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:01.570356 17416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:01.571501 17416 hybrid_clock.cc:648] HybridClock initialized: now 1786149481571489 us; error 27 us; skew 500 ppm
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149481, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:01.574064 17416 init.cc:377] Logged in from keytab as oryx/127.1.208.3@KRBTEST.COM (short username oryx)
I20260808 00:38:01.574544 17416 webserver.cc:460] Webserver started at http://127.1.208.3:46181/ using document root <none> and password file <none>
I20260808 00:38:01.574817 17416 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:01.574892 17416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:01.576357 17416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:38:01.576948 17441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.577133 17416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:38:01.577219 17416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "b1dd51c9fec54a32a5f1504e3db2916a"
format_stamp: "Formatted at 2026-08-08 00:38:00 on dist-test-slave-gjw7"
server_key: "a0dd10253b497adcba753431fe271ee0"
server_key_iv: "c3f23e0b23f675914b11a9be16acca48"
server_key_version: "encryptionkey@0"
I20260808 00:38:01.577641 17416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:01.585695 17416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:01.587407 17416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:01.587517 17416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:01.587689 17416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:01.587949 17416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:01.587978 17416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.587999 17416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:01.588016 17416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:01.593878 17416 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:35877
I20260808 00:38:01.593933 17647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:35877 every 8 connection(s)
I20260808 00:38:01.594259 17416 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
W20260808 00:38:01.601445 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.594962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:37015 (local address 127.1.208.62:37481)
0808 00:38:01.595027 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.595029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.595230 (+ 201us) server_negotiation.cc:407] Connection header received
0808 00:38:01.595786 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.595787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.595801 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.595859 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.596848 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.597399 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.598383 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.598585 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.598642 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.601379 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:37015: BlockingRecv error: recv got EOF from 127.1.208.3:37015 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:38:01.601534 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.594943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:37015)
0808 00:38:01.595109 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.595146 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.595682 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.595957 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.595961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.596544 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.596742 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.596747 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.597483 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.597484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.598293 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.598296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.598656 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.598657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.598774 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.599220 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.599229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.601238 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
W20260808 00:38:01.601735 17650 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37481 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.208.62:37481: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
I20260808 00:38:01.602175 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 17416
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.606176 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.601928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:34581)
0808 00:38:01.601972 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.601982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.602019 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.602340 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.602341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.602484 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.602538 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.602541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.603164 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.603165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.603747 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.603750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.604049 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.604050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.604111 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.604481 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.604486 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.606114 (+ 1628us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:38:01.606253 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.602124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:34581 (local address 127.1.208.62:37481)
0808 00:38:01.602165 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.602167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.602175 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:38:01.602198 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.602199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.602209 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.602264 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.602608 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.603095 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.603818 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.603994 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.604046 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.606200 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:34581: BlockingRecv error: recv got EOF from 127.1.208.3:34581 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.609207 17658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.602652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45530 (local address 127.1.208.62:37481)
0808 00:38:01.602761 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.602763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.602881 (+ 118us) server_negotiation.cc:407] Connection header received
0808 00:38:01.602901 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.602901 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.602911 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.602959 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.603580 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.604063 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.604804 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.605077 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.605143 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.606240 (+ 1097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:01.606246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:01.606246 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:01.606260 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:01.607803 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:01.608135 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:01.608138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:01.608138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:01.608266 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:01.608475 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:01.608476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:01.608476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:01.608707 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:01.608807 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:01.609002 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 00:38:01.609114 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
Aug 08 00:38:01 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:01.610342 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:01.606407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:55599)
0808 00:38:01.606442 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:01.606450 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:01.606482 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:01.606579 (+ 97us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:01.606580 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:01.606692 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:01.606717 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.606718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.607263 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.607263 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.607732 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:01.607733 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.608030 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.608031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:01.608081 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.608452 (+ 371us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:01.608457 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:01.610282 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 00:38:01.610409 17188 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:01.606442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55599 (local address 127.1.208.62:37481)
0808 00:38:01.606472 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:01.606473 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:01.606480 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:38:01.606525 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:01.606526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:01.606533 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:01.606577 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:01.606777 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.607200 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.607799 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:01.607976 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:01.608026 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:01.610362 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:55599: BlockingRecv error: recv got EOF from 127.1.208.3:55599 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 00:38:01.610479 17650 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:02.358062 17659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:02.353364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:48605)
0808 00:38:02.353497 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:02.353508 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:02.353576 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:02.353714 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:02.353715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:02.353887 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:02.353953 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.353955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.354624 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.354625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.355161 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.355162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.355524 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.355525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.355561 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.356000 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:02.356005 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:02.357989 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":63,"threads_started":1}
W20260808 00:38:02.358174 17662 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:02.353364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:48605 (local address 127.1.208.62:37481)
0808 00:38:02.353521 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:02.353523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:02.353564 (+ 41us) server_negotiation.cc:407] Connection header received
0808 00:38:02.353626 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:02.353627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:02.353639 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:02.353699 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:02.354024 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.354564 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.355267 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.355469 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.355522 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.358094 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:48605: BlockingRecv error: recv got EOF from 127.1.208.1:48605 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:02.488541 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:02.484043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:60111)
0808 00:38:02.484170 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:02.484181 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:02.484238 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:02.484360 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:02.484361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:02.484537 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:02.484595 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.484597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.485273 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.485274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.485847 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.485849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.486178 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.486178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.486217 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.486614 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:02.486618 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:02.488461 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":72,"threads_started":1}
W20260808 00:38:02.488610 17662 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:02.484102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:60111 (local address 127.1.208.62:37481)
0808 00:38:02.484162 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:02.484164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:02.484231 (+ 67us) server_negotiation.cc:407] Connection header received
0808 00:38:02.484280 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:02.484281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:02.484292 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:02.484350 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:02.484692 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.485193 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.485914 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.486113 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.486174 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.488555 (+ 2381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:60111: BlockingRecv error: recv got EOF from 127.1.208.2:60111 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:02 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:02.615299 17664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:02.610858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:36629)
0808 00:38:02.610985 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:02.610996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:02.611061 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:02.611191 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:02.611192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:02.611372 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:02.611438 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.611440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.612081 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.612082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.612631 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:02.612633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.612977 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.612978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:02.613036 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.613527 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:02.613532 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:02.615208 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260808 00:38:02.615363 17662 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:02.610910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:36629 (local address 127.1.208.62:37481)
0808 00:38:02.610957 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:02.610960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:02.611056 (+ 96us) server_negotiation.cc:407] Connection header received
0808 00:38:02.611114 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:02.611115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:02.611127 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:02.611178 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:02.611515 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.612005 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.612714 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:02.612912 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:02.612967 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:02.615307 (+ 2340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:36629: BlockingRecv error: recv got EOF from 127.1.208.3:36629 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:03.362843 17667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:03.358514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:51353)
0808 00:38:03.358659 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:03.358671 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:03.358736 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:03.358896 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:03.358898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:03.359072 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:03.359127 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.359128 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.359743 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.359744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.360262 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.360263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.360583 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.360583 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.360621 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.361032 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:03.361036 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:03.362781 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
W20260808 00:38:03.362996 17668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:03.358619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:51353 (local address 127.1.208.62:37481)
0808 00:38:03.358731 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:03.358733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:03.358742 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:03.358794 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:03.358795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:03.358806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:03.358862 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:03.359185 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.359692 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.360324 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.360531 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.360578 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.362874 (+ 2296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:51353: BlockingRecv error: recv got EOF from 127.1.208.1:51353 (error 108)
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":56,"threads_started":1}
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:03.493224 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:03.488983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:46829)
0808 00:38:03.489124 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:03.489136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:03.489195 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:03.489381 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:03.489382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:03.489573 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:03.489624 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.489626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.490226 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.490226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.490785 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.490788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.491072 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.491072 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.491108 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.491512 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:03.491516 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:03.493154 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":62,"threads_started":1}
W20260808 00:38:03.493327 17668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:03.489037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:46829 (local address 127.1.208.62:37481)
0808 00:38:03.489090 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:03.489093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:03.489187 (+ 94us) server_negotiation.cc:407] Connection header received
0808 00:38:03.489251 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:03.489252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:03.489262 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:03.489358 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:03.489679 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.490173 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.490845 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.491023 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.491067 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.493248 (+ 2181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:46829: BlockingRecv error: recv got EOF from 127.1.208.2:46829 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:03 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:03.620174 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:03.615783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:55267)
0808 00:38:03.615958 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:03.615969 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:03.616035 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:03.616140 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:03.616141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:03.616309 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:03.616359 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.616360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.617012 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.617012 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.617577 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:03.617579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.617958 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.617960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:03.618012 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.618443 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:03.618448 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:03.620107 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":79,"threads_started":1}
W20260808 00:38:03.620255 17668 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:03.615902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55267 (local address 127.1.208.62:37481)
0808 00:38:03.615963 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:03.615965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:03.616025 (+ 60us) server_negotiation.cc:407] Connection header received
0808 00:38:03.616075 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:03.616076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:03.616087 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:03.616133 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:03.616411 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.616940 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.617674 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:03.617872 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:03.617918 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:03.620194 (+ 2276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:55267: BlockingRecv error: recv got EOF from 127.1.208.3:55267 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:04.371748 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:04.363320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:50893)
0808 00:38:04.363475 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:04.363487 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:04.363536 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:04.363671 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:04.363672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:04.363841 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:04.363892 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.363894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.364490 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.364490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.365018 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.365020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.365369 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.365370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.365409 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.369999 (+ 4590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:04.370003 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:04.371667 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
W20260808 00:38:04.371845 17673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:04.363384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:50893 (local address 127.1.208.62:37481)
0808 00:38:04.363512 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:04.363514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:04.363522 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:38:04.363585 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:04.363586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:04.363596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:04.363660 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:04.363942 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.364436 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.365082 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.365278 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.365361 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.371781 (+ 6420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:50893: BlockingRecv error: recv got EOF from 127.1.208.1:50893 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:04.498169 17674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:04.493707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:36473)
0808 00:38:04.493934 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:04.493945 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:04.494024 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:04.494187 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:04.494188 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:04.494368 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:04.494420 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.494422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.495057 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.495059 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.495595 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.495597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.495943 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.495944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.496022 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.496428 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:04.496432 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:04.498081 (+ 1649us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":73,"threads_started":1}
W20260808 00:38:04.498246 17673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:04.493790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:36473 (local address 127.1.208.62:37481)
0808 00:38:04.493857 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:04.493859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:04.494016 (+ 157us) server_negotiation.cc:407] Connection header received
0808 00:38:04.494084 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:04.494085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:04.494100 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:04.494152 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:04.494495 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.494986 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.495672 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.495867 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.495936 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.498191 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:36473: BlockingRecv error: recv got EOF from 127.1.208.2:36473 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:04 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:04.625960 17675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:04.620693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:54655)
0808 00:38:04.620859 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:04.620873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:04.620957 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:04.621108 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:04.621109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:04.621348 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:04.621409 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.621412 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.622077 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.622078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.622628 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:04.622630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.622995 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.622997 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:04.623062 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.623572 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:04.623577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:04.625863 (+ 2286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":95,"threads_started":1}
W20260808 00:38:04.626032 17673 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:04.620809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:54655 (local address 127.1.208.62:37481)
0808 00:38:04.620892 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:04.620895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:04.620946 (+ 51us) server_negotiation.cc:407] Connection header received
0808 00:38:04.621023 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:04.621025 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:04.621040 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:04.621090 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:04.621486 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.621996 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.622706 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:04.622906 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:04.622964 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:04.625974 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:54655: BlockingRecv error: recv got EOF from 127.1.208.3:54655 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:05.376809 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:05.372297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:53811)
0808 00:38:05.372487 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:05.372499 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:05.372580 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:05.372710 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:05.372711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:05.372881 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:05.372934 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.372935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.373605 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.373606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.374120 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.374121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.374428 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.374428 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.374465 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.374930 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:05.374937 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:05.376747 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
W20260808 00:38:05.376911 17681 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:05.372371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:53811 (local address 127.1.208.62:37481)
0808 00:38:05.372522 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:05.372524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:05.372562 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:38:05.372632 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:05.372633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:05.372644 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:05.372698 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:05.372982 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.373549 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.374176 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.374364 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.374408 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.376842 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:53811: BlockingRecv error: recv got EOF from 127.1.208.1:53811 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:05.503144 17682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:05.498716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:60761)
0808 00:38:05.498852 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:05.498864 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:05.498937 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:05.499081 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:05.499082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:05.499260 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:05.499312 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.499314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.499916 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.499916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.500460 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.500461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.500754 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.500755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.500791 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.501207 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:05.501211 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:05.503078 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
W20260808 00:38:05.503249 17681 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:05.498793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:60761 (local address 127.1.208.62:37481)
0808 00:38:05.498856 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:05.498858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:05.498927 (+ 69us) server_negotiation.cc:407] Connection header received
0808 00:38:05.498995 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:05.498995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:05.499007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:05.499069 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:05.499365 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.499858 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.500522 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.500704 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.500751 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.503185 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:60761: BlockingRecv error: recv got EOF from 127.1.208.2:60761 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:05 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:05.630710 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:05.626474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:35619)
0808 00:38:05.626642 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:05.626653 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:05.626715 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:05.626836 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:05.626837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:05.627009 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:05.627058 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.627060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.627692 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.627694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.628224 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:05.628226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.628522 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.628523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:05.628561 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.629010 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:05.629014 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:05.630647 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":92,"threads_started":1}
W20260808 00:38:05.630815 17681 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:05.626562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35619 (local address 127.1.208.62:37481)
0808 00:38:05.626628 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:05.626630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:05.626706 (+ 76us) server_negotiation.cc:407] Connection header received
0808 00:38:05.626770 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:05.626771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:05.626783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:05.626821 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:05.627107 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.627618 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.628285 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:05.628472 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:05.628515 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:05.630750 (+ 2235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:35619: BlockingRecv error: recv got EOF from 127.1.208.3:35619 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:06.382347 17684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:06.377484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:38707)
0808 00:38:06.377639 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:06.377653 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:06.377726 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:06.377870 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:06.377870 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:06.378067 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:06.378120 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.378121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.378796 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.378796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.379364 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.379366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.379688 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.379689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.379727 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.380231 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:06.380236 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:06.382278 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":76,"threads_started":1}
W20260808 00:38:06.382565 17685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:06.377556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:38707 (local address 127.1.208.62:37481)
0808 00:38:06.377696 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:06.377699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:06.377709 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:38:06.377779 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:06.377780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:06.377792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:06.377856 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:06.378183 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.378728 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.379436 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.379633 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.379684 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.382474 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:38707: BlockingRecv error: recv got EOF from 127.1.208.1:38707 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":79,"threads_started":1}
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:06.508334 17686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:06.503636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:40433)
0808 00:38:06.503781 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:06.503792 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:06.503854 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:06.503984 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:06.503985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:06.504156 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:06.504214 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.504216 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.504935 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.504937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.505521 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.505524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.505881 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.505882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.505936 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.506435 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:06.506441 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:06.508248 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
W20260808 00:38:06.508426 17685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:06.503749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:40433 (local address 127.1.208.62:37481)
0808 00:38:06.503812 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:06.503814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:06.503838 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:38:06.503905 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:06.503905 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:06.503916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:06.503964 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:06.504301 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.504851 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.505605 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.505813 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.505870 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.508370 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:40433: BlockingRecv error: recv got EOF from 127.1.208.2:40433 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:06 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:06.636211 17688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:06.631310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:40139)
0808 00:38:06.631452 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:06.631469 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:06.631549 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:06.631696 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:06.631697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:06.631884 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:06.631936 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.631938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.632635 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.632636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.633165 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:06.633167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.633580 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.633581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:06.633636 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.634125 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:06.634130 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:06.636127 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
W20260808 00:38:06.636307 17685 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:06.631465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:40139 (local address 127.1.208.62:37481)
0808 00:38:06.631555 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:06.631557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:06.631586 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:38:06.631618 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:06.631619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:06.631634 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:06.631677 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:06.632016 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.632543 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.633255 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:06.633510 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:06.633573 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:06.636247 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:40139: BlockingRecv error: recv got EOF from 127.1.208.3:40139 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:07.387759 17689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:07.382857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:44675)
0808 00:38:07.383011 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:07.383022 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:07.383087 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:07.383232 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:07.383232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:07.383417 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:07.383468 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.383470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.384105 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.384106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.384653 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.384654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.385081 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.385083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.385159 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.385761 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:07.385767 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:07.387693 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
W20260808 00:38:07.387935 17691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:07.382943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:44675 (local address 127.1.208.62:37481)
0808 00:38:07.383064 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:07.383066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:07.383077 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:38:07.383151 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:07.383152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:07.383163 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:07.383218 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:07.383523 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.384032 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.384746 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.384965 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.385036 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.387851 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:44675: BlockingRecv error: recv got EOF from 127.1.208.1:44675 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:07.513666 17695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:07.508914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:53661)
0808 00:38:07.509077 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:07.509089 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:07.509162 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:07.509316 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:07.509317 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:07.509543 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:07.509599 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.509602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.510310 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.510311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.510840 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.510841 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.511166 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.511166 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.511206 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.511705 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:07.511710 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:07.513579 (+ 1869us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
W20260808 00:38:07.513752 17691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:07.508987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:53661 (local address 127.1.208.62:37481)
0808 00:38:07.509081 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:07.509083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:07.509155 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:38:07.509214 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:07.509215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:07.509227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:07.509275 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:07.509669 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.510234 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.510920 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.511111 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.511162 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.513698 (+ 2536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:53661: BlockingRecv error: recv got EOF from 127.1.208.2:53661 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:07 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:07.641665 17699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:07.636800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:42927)
0808 00:38:07.636945 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:07.636957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:07.637023 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:07.637158 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:07.637159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:07.637389 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:07.637443 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.637446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.638128 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.638130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.638851 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:07.638855 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.639174 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.639174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:07.639213 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.639732 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:07.639738 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:07.641599 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
W20260808 00:38:07.641772 17691 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:07.636890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:42927 (local address 127.1.208.62:37481)
0808 00:38:07.636988 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:07.636990 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:07.637006 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:38:07.637084 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:07.637085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:07.637098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:07.637144 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:07.637506 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.638007 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.638919 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:07.639116 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:07.639169 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:07.641704 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:42927: BlockingRecv error: recv got EOF from 127.1.208.3:42927 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:08.392652 17707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:08.388258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:45283)
0808 00:38:08.388397 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:08.388408 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:08.388474 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:08.388605 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:08.388606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:08.388776 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:08.388828 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.388829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.389480 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.389481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.390025 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.390027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.390385 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.390386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.390441 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.390952 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:08.390957 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:08.392590 (+ 1633us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
W20260808 00:38:08.392769 17708 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:08.388346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:45283 (local address 127.1.208.62:37481)
0808 00:38:08.388457 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:08.388459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:08.388469 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:38:08.388523 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:08.388523 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:08.388534 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:08.388588 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:08.388881 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.389426 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.390093 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.390325 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.390379 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.392683 (+ 2304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:45283: BlockingRecv error: recv got EOF from 127.1.208.1:45283 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":58,"threads_started":1}
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:08.518744 17710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:08.514234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:55901)
0808 00:38:08.514389 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:08.514401 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:08.514494 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:08.514620 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:08.514620 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:08.514790 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:08.514844 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.514846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.515460 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.515461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.516008 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.516010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.516299 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.516299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.516335 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.516800 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:08.516806 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:08.518644 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
W20260808 00:38:08.518826 17708 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:08.514298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:55901 (local address 127.1.208.62:37481)
0808 00:38:08.514400 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:08.514402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:08.514484 (+ 82us) server_negotiation.cc:407] Connection header received
0808 00:38:08.514544 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:08.514545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:08.514555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:08.514608 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:08.514895 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.515400 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.516067 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.516248 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.516292 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.518770 (+ 2478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:55901: BlockingRecv error: recv got EOF from 127.1.208.2:55901 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:08 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:08.646912 17711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:08.642190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:55625)
0808 00:38:08.642384 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:08.642398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:08.642477 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:08.642607 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:08.642608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:08.642775 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:08.642830 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.642832 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.643503 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.643504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.644053 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:08.644055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.644404 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.644405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:08.644443 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.644921 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:08.644925 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:08.646829 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":120,"threads_started":1}
W20260808 00:38:08.646996 17708 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:08.642251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:55625 (local address 127.1.208.62:37481)
0808 00:38:08.642326 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:08.642329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:08.642489 (+ 160us) server_negotiation.cc:407] Connection header received
0808 00:38:08.642535 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:08.642536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:08.642551 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:08.642592 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:08.642882 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.643412 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.644126 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:08.644349 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:08.644399 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:08.646939 (+ 2540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:55625: BlockingRecv error: recv got EOF from 127.1.208.3:55625 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:09.401467 17712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:09.393121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:56809)
0808 00:38:09.393280 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:09.393309 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:09.393381 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:09.393534 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:09.393535 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:09.393704 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:09.393754 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.393756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.394359 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.394360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.394884 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.394885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.395156 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.395157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.395192 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.399529 (+ 4337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:09.399534 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:09.401405 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":74,"threads_started":1}
W20260808 00:38:09.401589 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:09.393261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:56809 (local address 127.1.208.62:37481)
0808 00:38:09.393425 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:09.393427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:09.393435 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:38:09.393456 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:09.393457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:09.393467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:09.393523 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:09.393805 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.394309 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.394936 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.395113 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.395154 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.401507 (+ 6353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:56809: BlockingRecv error: recv got EOF from 127.1.208.1:56809 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":98,"threads_started":1}
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:09.523488 17717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:09.519257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:40509)
0808 00:38:09.519388 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:09.519399 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:09.519471 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:09.519587 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:09.519587 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:09.519749 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:09.519801 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.519803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.520542 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.520542 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.521063 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.521064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.521380 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.521382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.521440 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.521839 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:09.521844 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:09.523409 (+ 1565us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
W20260808 00:38:09.523573 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:09.519324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:40509 (local address 127.1.208.62:37481)
0808 00:38:09.519392 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:09.519394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:09.519451 (+ 57us) server_negotiation.cc:407] Connection header received
0808 00:38:09.519516 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:09.519517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:09.519529 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:09.519574 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:09.519849 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.520488 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.521115 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.521317 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.521376 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.523520 (+ 2144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:40509: BlockingRecv error: recv got EOF from 127.1.208.2:40509 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:09 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:09.651876 17719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:09.647432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:49607)
0808 00:38:09.647555 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:09.647566 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:09.647631 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:09.647763 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:09.647764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:09.647953 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:09.648002 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.648005 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.648692 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.648693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.649212 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:09.649214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.649591 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.649592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:09.649635 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.650025 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:09.650029 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:09.651795 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260808 00:38:09.651949 17713 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:09.647490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:49607 (local address 127.1.208.62:37481)
0808 00:38:09.647545 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:09.647547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:09.647624 (+ 77us) server_negotiation.cc:407] Connection header received
0808 00:38:09.647678 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:09.647679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:09.647701 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:09.647751 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:09.648066 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.648627 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.649279 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:09.649526 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:09.649572 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:09.651893 (+ 2321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:49607: BlockingRecv error: recv got EOF from 127.1.208.3:49607 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:10.406368 17720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:10.401875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:34129)
0808 00:38:10.402017 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:10.402028 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:10.402074 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:10.402221 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:10.402222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:10.402410 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:10.402472 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.402475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.403132 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.403132 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.403768 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.403771 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.404124 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.404125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.404164 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.404580 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:10.404584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:10.406305 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":83,"threads_started":1}
W20260808 00:38:10.406519 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:10.401950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34129 (local address 127.1.208.62:37481)
0808 00:38:10.402052 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:10.402054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:10.402077 (+ 23us) server_negotiation.cc:407] Connection header received
0808 00:38:10.402121 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:10.402122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:10.402132 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:10.402184 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:10.402540 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.403071 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.403857 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.404069 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.404126 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.406439 (+ 2313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:34129: BlockingRecv error: recv got EOF from 127.1.208.1:34129 (error 108)
Metrics: {"server-negotiator.queue_time_us":79,"thread_start_us":54,"threads_started":1}
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:10.528373 17724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:10.523923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:49863)
0808 00:38:10.524072 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:10.524083 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:10.524155 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:10.524280 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:10.524281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:10.524487 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:10.524537 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.524540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.525180 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.525181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.525731 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.525733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.526067 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.526068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.526124 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.526549 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:10.526555 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:10.528313 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":73,"threads_started":1}
W20260808 00:38:10.528474 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:10.523987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:49863 (local address 127.1.208.62:37481)
0808 00:38:10.524040 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:10.524043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:10.524146 (+ 103us) server_negotiation.cc:407] Connection header received
0808 00:38:10.524209 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:10.524210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:10.524223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:10.524268 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:10.524613 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.525107 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.525812 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.526006 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.526061 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.528410 (+ 2349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:49863: BlockingRecv error: recv got EOF from 127.1.208.2:49863 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:10 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:10.656466 17725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:10.652295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:37455)
0808 00:38:10.652462 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:10.652472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:10.652526 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:10.652642 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:10.652643 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:10.652806 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:10.652865 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.652867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.653485 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.653486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.654006 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:10.654008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.654291 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.654292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:10.654327 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.654716 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:10.654720 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:10.656402 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
W20260808 00:38:10.656554 17722 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:10.652366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:37455 (local address 127.1.208.62:37481)
0808 00:38:10.652417 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:10.652419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:10.652520 (+ 101us) server_negotiation.cc:407] Connection header received
0808 00:38:10.652567 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:10.652568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:10.652579 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:10.652631 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:10.652927 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.653431 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.654068 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:10.654246 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:10.654287 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:10.656490 (+ 2203us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:37455: BlockingRecv error: recv got EOF from 127.1.208.3:37455 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:11.411216 17730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:11.406769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:38123)
0808 00:38:11.406932 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:11.406945 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:11.407019 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:11.407144 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:11.407145 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:11.407328 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:11.407390 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.407392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.408017 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.408017 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.408563 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.408565 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.408871 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.408871 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.408911 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.409421 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:11.409426 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:11.411148 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
W20260808 00:38:11.411304 17731 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:11.406847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:38123 (local address 127.1.208.62:37481)
0808 00:38:11.406963 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:11.406965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:11.407009 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:38:11.407073 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:11.407074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:11.407084 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:11.407138 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:11.407447 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.407961 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.408629 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.408811 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.408861 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.411239 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:38123: BlockingRecv error: recv got EOF from 127.1.208.1:38123 (error 108)
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":54,"threads_started":1}
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:11.533427 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:11.528832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:56005)
0808 00:38:11.528998 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:11.529012 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:11.529096 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:11.529269 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:11.529271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:11.529458 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:11.529512 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.529514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.530163 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.530164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.530732 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.530734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.531035 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.531036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.531077 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.531597 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:11.531602 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:11.533344 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":61,"threads_started":1}
W20260808 00:38:11.533573 17731 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:11.528964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:56005 (local address 127.1.208.62:37481)
0808 00:38:11.529020 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:11.529022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:11.529075 (+ 53us) server_negotiation.cc:407] Connection header received
0808 00:38:11.529152 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:11.529153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:11.529166 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:11.529202 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:11.529578 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.530086 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.530796 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.530983 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.531026 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.533444 (+ 2418us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:56005: BlockingRecv error: recv got EOF from 127.1.208.2:56005 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:11 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:11.661152 17735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:11.656980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:35987)
0808 00:38:11.657155 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:11.657167 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:11.657236 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:11.657413 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:11.657413 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:11.657585 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:11.657637 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.657639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.658258 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.658259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.658792 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:11.658794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.659092 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.659092 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:11.659129 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.659534 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:11.659539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:11.661093 (+ 1554us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
W20260808 00:38:11.661252 17731 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:11.657119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35987 (local address 127.1.208.62:37481)
0808 00:38:11.657180 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:11.657182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:11.657227 (+ 45us) server_negotiation.cc:407] Connection header received
0808 00:38:11.657313 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:11.657315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:11.657349 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:11.657391 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:11.657691 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.658193 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.658859 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:11.659041 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:11.659087 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:11.661180 (+ 2093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:35987: BlockingRecv error: recv got EOF from 127.1.208.3:35987 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:12.415858 17740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:12.411652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:47539)
0808 00:38:12.411772 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:12.411783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:12.411844 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:12.411980 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:12.411981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:12.412152 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:12.412202 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.412203 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.412817 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.412817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.413362 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.413364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.413649 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.413649 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.413684 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.414076 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:12.414080 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:12.415787 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":71,"threads_started":1}
W20260808 00:38:12.415978 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:12.411691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:47539 (local address 127.1.208.62:37481)
0808 00:38:12.411820 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:12.411822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:12.411831 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:12.411899 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:12.411900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:12.411910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:12.411964 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:12.412252 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.412763 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.413423 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.413603 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.413646 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.415884 (+ 2238us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:47539: BlockingRecv error: recv got EOF from 127.1.208.1:47539 (error 108)
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":68,"threads_started":1}
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:12.538412 17742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:12.533934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:52477)
0808 00:38:12.534080 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:12.534093 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:12.534171 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:12.534305 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:12.534306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:12.534519 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:12.534567 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.534569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.535232 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.535233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.535788 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.535790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.536152 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.536153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.536210 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.536635 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:12.536639 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:12.538294 (+ 1655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
W20260808 00:38:12.538455 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:12.534015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:52477 (local address 127.1.208.62:37481)
0808 00:38:12.534083 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:12.534086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:12.534158 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:38:12.534227 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:12.534228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:12.534240 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:12.534287 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:12.534644 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.535156 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.535880 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.536089 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.536147 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.538400 (+ 2253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:52477: BlockingRecv error: recv got EOF from 127.1.208.2:52477 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:12 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:12.665966 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:12.661579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:53951)
0808 00:38:12.661714 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:12.661726 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:12.661803 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:12.661944 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:12.661945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:12.662130 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:12.662179 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.662181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.662850 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.662851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.663387 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:12.663389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.663720 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.663722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:12.663777 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.664192 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:12.664197 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:12.665888 (+ 1691us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":72,"threads_started":1}
W20260808 00:38:12.666046 17741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:12.661645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:53951 (local address 127.1.208.62:37481)
0808 00:38:12.661717 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:12.661719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:12.661790 (+ 71us) server_negotiation.cc:407] Connection header received
0808 00:38:12.661860 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:12.661861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:12.661876 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:12.661924 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:12.662256 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.662771 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.663470 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:12.663660 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:12.663715 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:12.665985 (+ 2270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:53951: BlockingRecv error: recv got EOF from 127.1.208.3:53951 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:13.420711 17748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:13.416301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:38709)
0808 00:38:13.416462 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:13.416472 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:13.416535 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:13.416657 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:13.416658 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:13.416844 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:13.416897 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.416899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.417565 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.417566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.418102 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.418104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.418416 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.418417 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.418453 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.418863 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:13.418867 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:13.420643 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
W20260808 00:38:13.420796 17750 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:13.416362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:38709 (local address 127.1.208.62:37481)
0808 00:38:13.416474 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:13.416476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:13.416518 (+ 42us) server_negotiation.cc:407] Connection header received
0808 00:38:13.416576 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:13.416577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:13.416588 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:13.416648 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:13.416965 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.417496 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.418166 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.418365 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.418413 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.420725 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:38709: BlockingRecv error: recv got EOF from 127.1.208.1:38709 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":53,"threads_started":1}
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:13.543217 17751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:13.538885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:57231)
0808 00:38:13.539018 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:13.539031 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:13.539103 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:13.539230 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:13.539231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:13.539428 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:13.539477 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.539479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.540162 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.540163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.540695 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.540697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.541026 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.541027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.541086 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.541522 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:13.541526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:13.543123 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260808 00:38:13.543278 17750 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:13.538967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:57231 (local address 127.1.208.62:37481)
0808 00:38:13.539028 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:13.539031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:13.539079 (+ 48us) server_negotiation.cc:407] Connection header received
0808 00:38:13.539154 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:13.539155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:13.539167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:13.539217 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:13.539546 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.540069 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.540766 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.540965 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.541020 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.543225 (+ 2205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:57231: BlockingRecv error: recv got EOF from 127.1.208.2:57231 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:13 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:13.670630 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:13.666419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:45179)
0808 00:38:13.666571 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:13.666582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:13.666638 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:13.666751 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:13.666752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:13.666938 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:13.666987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.666989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.667582 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.667583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.668103 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:13.668104 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.668364 (+ 260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.668364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:13.668402 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.668804 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:13.668808 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:13.670533 (+ 1725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":82,"threads_started":1}
W20260808 00:38:13.670691 17750 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:13.666494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:45179 (local address 127.1.208.62:37481)
0808 00:38:13.666549 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:13.666551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:13.666630 (+ 79us) server_negotiation.cc:407] Connection header received
0808 00:38:13.666679 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:13.666680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:13.666692 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:13.666738 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:13.667035 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.667529 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.668152 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:13.668320 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:13.668360 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:13.670635 (+ 2275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:45179: BlockingRecv error: recv got EOF from 127.1.208.3:45179 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:14.429455 17755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:14.421114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:46087)
0808 00:38:14.421264 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:14.421277 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:14.421360 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:14.421476 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:14.421477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:14.421646 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:14.421697 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.421698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.422295 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.422295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.422822 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.422823 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.423107 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.423108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.423142 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.427542 (+ 4400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:14.427547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:14.429373 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
W20260808 00:38:14.429533 17757 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:14.421174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:46087 (local address 127.1.208.62:37481)
0808 00:38:14.421367 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:14.421369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:14.421378 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:14.421401 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:14.421402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:14.421412 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:14.421465 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:14.421747 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.422241 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.422877 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.423061 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.423104 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.429463 (+ 6359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:46087: BlockingRecv error: recv got EOF from 127.1.208.1:46087 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:14.548113 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:14.543751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:36487)
0808 00:38:14.543890 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:14.543900 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:14.543958 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:14.544084 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:14.544086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:14.544257 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:14.544307 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.544308 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.545061 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.545062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.545669 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.545671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.545971 (+ 300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.545971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.546007 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.546405 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:14.546409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:14.548052 (+ 1643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":74,"threads_started":1}
W20260808 00:38:14.548199 17757 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:14.543899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:36487 (local address 127.1.208.62:37481)
0808 00:38:14.543974 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:14.543976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:14.543985 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:14.544014 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:14.544015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:14.544026 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:14.544071 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:14.544377 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.544990 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.545730 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.545919 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.545968 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.548137 (+ 2169us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:36487: BlockingRecv error: recv got EOF from 127.1.208.2:36487 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:14 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:14.675626 17759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:14.671121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:35633)
0808 00:38:14.671285 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:14.671297 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:14.671365 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:14.671490 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:14.671491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:14.671689 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:14.671739 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.671741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.672460 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.672461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.673023 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:14.673025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.673415 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.673416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:14.673475 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.673880 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:14.673884 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:14.675554 (+ 1670us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":99,"threads_started":1}
W20260808 00:38:14.675696 17757 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:14.671187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35633 (local address 127.1.208.62:37481)
0808 00:38:14.671262 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:14.671265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:14.671346 (+ 81us) server_negotiation.cc:407] Connection header received
0808 00:38:14.671417 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:14.671418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:14.671432 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:14.671484 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:14.671811 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.672393 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.673102 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:14.673341 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:14.673395 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:14.675643 (+ 2248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:35633: BlockingRecv error: recv got EOF from 127.1.208.3:35633 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:15.434762 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:15.429986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:34609)
0808 00:38:15.430124 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:15.430135 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:15.430189 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:15.430318 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:15.430319 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:15.430507 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:15.430559 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.430560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.431172 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.431173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.431733 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.431736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.432198 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.432199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.432258 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.432762 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:15.432768 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:15.434696 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":69,"threads_started":1}
W20260808 00:38:15.434933 17764 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:15.430072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34609 (local address 127.1.208.62:37481)
0808 00:38:15.430195 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:15.430197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:15.430206 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:15.430233 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:15.430233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:15.430244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:15.430306 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:15.430614 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.431118 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.431835 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.432050 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.432124 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.434846 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:34609: BlockingRecv error: recv got EOF from 127.1.208.1:34609 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:15.553429 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:15.548628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:46037)
0808 00:38:15.548786 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:15.548798 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:15.548870 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:15.549008 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:15.549009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:15.549202 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:15.549315 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.549318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.549971 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.549972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.550548 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.550550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.550925 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.550926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.550972 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.551414 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:15.551418 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:15.553350 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
W20260808 00:38:15.553484 17764 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:15.548716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:46037 (local address 127.1.208.62:37481)
0808 00:38:15.548811 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:15.548813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:15.548853 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:38:15.548927 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:15.548928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:15.548938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:15.548987 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:15.549371 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.549899 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.550642 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.550852 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.550916 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.553433 (+ 2517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:46037: BlockingRecv error: recv got EOF from 127.1.208.2:46037 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:15 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.3@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:15.680655 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:15.676149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.3:50783)
0808 00:38:15.676294 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:15.676305 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:15.676377 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:15.676490 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:15.676491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:15.676668 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:15.676718 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.676719 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.677379 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.677379 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.677906 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:15.677909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.678203 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.678204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:15.678240 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.678699 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:15.678704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:15.680560 (+ 1856us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260808 00:38:15.680738 17764 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:15.676256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:50783 (local address 127.1.208.62:37481)
0808 00:38:15.676314 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:15.676316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:15.676372 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:38:15.676432 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:15.676433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:15.676445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:15.676482 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:15.676766 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.677271 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.677971 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:15.678151 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:15.678198 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:15.680678 (+ 2480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.3:50783: BlockingRecv error: recv got EOF from 127.1.208.3:50783 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 00:38:16 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:16 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.1@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:16.440346 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:16.435419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.1:53137)
0808 00:38:16.435600 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:16.435612 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:16.435666 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:16.435820 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:16.435821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:16.436014 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:16.436073 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.436075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.436919 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.436920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.437578 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.437581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.437967 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.437968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.438012 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.438485 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:16.438489 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:16.440267 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":89,"threads_started":1}
W20260808 00:38:16.440449 17769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:16.435545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:53137 (local address 127.1.208.62:37481)
0808 00:38:16.435679 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:16.435682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:16.435693 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:38:16.435723 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:16.435724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:16.435735 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:16.435807 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:16.436150 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.436845 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.437666 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.437885 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.437947 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.440369 (+ 2422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.1:53137: BlockingRecv error: recv got EOF from 127.1.208.1:53137 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
Aug 08 00:38:16 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
Aug 08 00:38:16 dist-test-slave-gjw7 krb5kdc[15871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.208.2@KRBTEST.COM for oryx/127.1.208.62@KRBTEST.COM, Server not found in Kerberos database
I20260808 00:38:16.558804 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:16.554041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37481 (local address 127.1.208.2:35475)
0808 00:38:16.554184 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:16.554197 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:16.554271 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:16.554423 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:16.554424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:16.554603 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:16.554652 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.554654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.555357 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.555357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.555888 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.555890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.556331 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.556332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.556394 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.556951 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:16.556955 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:16.558725 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.208.62:37481: Server oryx/127.1.208.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
W20260808 00:38:16.558887 17769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:16.554124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:35475 (local address 127.1.208.62:37481)
0808 00:38:16.554219 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:16.554221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:16.554279 (+ 58us) server_negotiation.cc:407] Connection header received
0808 00:38:16.554324 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:16.554325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:16.554337 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:16.554392 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:16.554747 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.555290 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.555994 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.556251 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.556324 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.558826 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.208.2:35475: BlockingRecv error: recv got EOF from 127.1.208.2:35475 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 00:38:16.610566 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 16912
I20260808 00:38:16.630851 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 17189
I20260808 00:38:16.644493 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 17416
I20260808 00:38:16.669731 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 16785
2026-08-08T00:38:16Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17703 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T00:38:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:38:16Z Disabled control of system clock
I20260808 00:38:16.701835 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43721
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:43721
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 00:38:16.784343 17777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:16.784562 17777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:16.784582 17777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:16.785766 17777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:38:16.785818 17777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:16.785832 17777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:16.785843 17777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:38:16.785857 17777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:38:16.787232 17777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:43721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17777
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:16.787436 17777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:16.787703 17777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:38:16.792533 17777 server_base.cc:1034] running on GCE node
W20260808 00:38:16.792477 17790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:16.792497 17788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:16.792467 17794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:16.793011 17777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:16.793372 17777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:16.794518 17777 hybrid_clock.cc:648] HybridClock initialized: now 1786149496794500 us; error 37 us; skew 500 ppm
I20260808 00:38:16.795789 17777 webserver.cc:460] Webserver started at http://127.1.208.62:34045/ using document root <none> and password file <none>
I20260808 00:38:16.796062 17777 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:16.796111 17777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:16.796252 17777 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:16.797416 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "287dbed865844cd0b0658cede718af7d"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "42472a3c0d6b84abcfad59c87171ed5f"
server_key_iv: "c074701f3163048e5ee6cb34f2231067"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.797773 17777 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "287dbed865844cd0b0658cede718af7d"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "42472a3c0d6b84abcfad59c87171ed5f"
server_key_iv: "c074701f3163048e5ee6cb34f2231067"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.799428 17777 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:38:16.800184 17815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:16.800391 17777 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:16.800482 17777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "287dbed865844cd0b0658cede718af7d"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "42472a3c0d6b84abcfad59c87171ed5f"
server_key_iv: "c074701f3163048e5ee6cb34f2231067"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.800554 17777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:16.813083 17777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:16.813452 17777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:16.813601 17777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:16.817749 17777 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:43721
I20260808 00:38:16.817795 17902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:43721 every 8 connection(s)
I20260808 00:38:16.818128 17777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:38:16.818836 17906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:16.821489 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d: Bootstrap starting.
I20260808 00:38:16.822073 17906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:16.822319 17906 log.cc:824] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:16.822981 17906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d: No bootstrap required, opened a new log
I20260808 00:38:16.824556 17906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } }
I20260808 00:38:16.824699 17906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:16.824721 17906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 287dbed865844cd0b0658cede718af7d, State: Initialized, Role: FOLLOWER
I20260808 00:38:16.824851 17906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } }
I20260808 00:38:16.824929 17906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:38:16.824957 17906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:38:16.824992 17906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:16.825527 17906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } }
I20260808 00:38:16.825618 17906 leader_election.cc:302] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 287dbed865844cd0b0658cede718af7d; no voters:
I20260808 00:38:16.825752 17906 leader_election.cc:288] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:38:16.825806 17909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:38:16.825985 17906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:38:16.826009 17909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [term 1 LEADER]: Becoming Leader. State: Replica: 287dbed865844cd0b0658cede718af7d, State: Running, Role: LEADER
I20260808 00:38:16.826186 17909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } }
I20260808 00:38:16.826381 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 17777
I20260808 00:38:16.826498 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:38:16.826733 1856 external_mini_cluster.cc:1428] Setting key 686d00162741ae81e58773e25b5bc775
I20260808 00:38:16.826807 17911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 287dbed865844cd0b0658cede718af7d. Latest consensus state: current_term: 1 leader_uuid: "287dbed865844cd0b0658cede718af7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } } }
I20260808 00:38:16.826882 17911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:16.827139 17925 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:38:16.827638 17910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "287dbed865844cd0b0658cede718af7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "287dbed865844cd0b0658cede718af7d" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43721 } } }
I20260808 00:38:16.827715 17910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:16.827754 17925 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:38:16.830261 17925 catalog_manager.cc:1347] Generated new cluster ID: 94e74599322044bcaa78002a038aab00
I20260808 00:38:16.830325 17925 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:38:16.836364 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:16.827281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52836 (local address 127.1.208.62:43721)
0808 00:38:16.828549 (+ 1268us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:16.828557 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:16.828593 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:38:16.828707 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:16.828714 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:16.828782 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:16.828936 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:16.829135 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.833604 (+ 4469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.834377 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.834805 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.834915 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.835651 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:16.835656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:16.835656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:16.835821 (+ 165us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:16.835856 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:16.835868 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:16.835938 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:16.836036 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 00:38:16.836199 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260808 00:38:16.838729 17925 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:38:16.839210 17925 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:38:16.845453 17925 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 287dbed865844cd0b0658cede718af7d: Generated new TSK 0
I20260808 00:38:16.845624 17925 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:38:16.849527 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--builtin_ntp_servers=127.1.208.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:38:16.942348 17951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:16.942574 17951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:16.942596 17951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:16.943789 17951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:16.943833 17951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:16.943858 17951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:38:16.945209 17951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:16.945499 17951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:16.945806 17951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:16.951577 17960 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:16.951601 17962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:16.951798 17951 server_base.cc:1034] running on GCE node
W20260808 00:38:16.951620 17959 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:16.952059 17951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:16.952294 17951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:16.953442 17951 hybrid_clock.cc:648] HybridClock initialized: now 1786149496953430 us; error 34 us; skew 500 ppm
I20260808 00:38:16.954692 17951 webserver.cc:460] Webserver started at http://127.1.208.1:41643/ using document root <none> and password file <none>
I20260808 00:38:16.954927 17951 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:16.954973 17951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:16.955159 17951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:16.956308 17951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "20f9c31a19fa4f97ad6b3b557f3bd03a"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "7bf4fa23845e39986109675c7ec52563"
server_key_iv: "4442f5156a6431b9b5274baa7d8c0290"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.956668 17951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "20f9c31a19fa4f97ad6b3b557f3bd03a"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "7bf4fa23845e39986109675c7ec52563"
server_key_iv: "4442f5156a6431b9b5274baa7d8c0290"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.958027 17951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:38:16.958753 17971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:16.958917 17951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:38:16.959007 17951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "20f9c31a19fa4f97ad6b3b557f3bd03a"
format_stamp: "Formatted at 2026-08-08 00:38:16 on dist-test-slave-gjw7"
server_key: "7bf4fa23845e39986109675c7ec52563"
server_key_iv: "4442f5156a6431b9b5274baa7d8c0290"
server_key_version: "encryptionkey@0"
I20260808 00:38:16.959076 17951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:16.977823 17951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:16.978163 17951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:16.978296 17951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:16.978529 17951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:16.978873 17951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:16.978914 17951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:16.978947 17951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:16.978971 17951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:16.984784 17951 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:46447
I20260808 00:38:16.984838 18172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:46447 every 8 connection(s)
I20260808 00:38:16.985174 17951 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:38:16.986347 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 17951
I20260808 00:38:16.986438 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:38:16.986621 1856 external_mini_cluster.cc:1428] Setting key 51ded009ae7413b24b234d7654ef0f49
I20260808 00:38:16.987618 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--builtin_ntp_servers=127.1.208.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 00:38:16.990744 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:16.985949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:53607 (local address 127.1.208.62:43721)
0808 00:38:16.986013 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:16.986014 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:16.986235 (+ 221us) server_negotiation.cc:407] Connection header received
0808 00:38:16.986798 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:16.986800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:16.986814 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:16.986899 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:16.987722 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.988217 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.989317 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.989561 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.989621 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.990514 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:16.990516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:16.990516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:16.990521 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:16.990539 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:16.990543 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:16.990589 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:16.990655 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 00:38:16.990682 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:38:16.991007 18177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:16.985907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43721 (local address 127.1.208.1:53607)
0808 00:38:16.986105 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:16.986148 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:16.986689 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:16.986987 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:16.986992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:16.987002 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:16.987627 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.987634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.988304 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.988305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.989216 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:16.989219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:16.989644 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:16.989644 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:16.989807 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:16.990360 (+ 553us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:16.990360 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:16.990402 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:16.990435 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:16.990592 (+ 157us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:16.990598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:16.990601 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:38:16.990662 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:38:16.990774 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260808 00:38:16.991587 18174 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43721
I20260808 00:38:16.991694 18174 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:16.991901 18174 heartbeater.cc:511] Master 127.1.208.62:43721 requested a full tablet report, sending...
I20260808 00:38:16.992467 17843 ts_manager.cc:194] Registered new tserver with Master: 20f9c31a19fa4f97ad6b3b557f3bd03a (127.1.208.1:46447)
I20260808 00:38:16.993091 17843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.1:53607
W20260808 00:38:17.075268 18178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.075487 18178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.075516 18178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.076685 18178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.076725 18178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.076756 18178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:38:17.078128 18178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.078365 18178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.078665 18178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.083478 18187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.083504 18186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.083503 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.083678 18178 server_base.cc:1034] running on GCE node
I20260808 00:38:17.083910 18178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.084115 18178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.085245 18178 hybrid_clock.cc:648] HybridClock initialized: now 1786149497085232 us; error 28 us; skew 500 ppm
I20260808 00:38:17.086548 18178 webserver.cc:460] Webserver started at http://127.1.208.2:38939/ using document root <none> and password file <none>
I20260808 00:38:17.086762 18178 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.086825 18178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.086963 18178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.088047 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "8c89ac6b7e7541608888df941f0d05ce"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "e0099f6867fb563fb2c2842df17ef8c4"
server_key_iv: "26e38f71f888bc0d47dccff9c303e5f9"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.088380 18178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "8c89ac6b7e7541608888df941f0d05ce"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "e0099f6867fb563fb2c2842df17ef8c4"
server_key_iv: "26e38f71f888bc0d47dccff9c303e5f9"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.089649 18178 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:38:17.090312 18206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.090467 18178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.090551 18178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "8c89ac6b7e7541608888df941f0d05ce"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "e0099f6867fb563fb2c2842df17ef8c4"
server_key_iv: "26e38f71f888bc0d47dccff9c303e5f9"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.090611 18178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.098822 18178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.099084 18178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.099202 18178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.099399 18178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:17.099710 18178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:17.099743 18178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.099771 18178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:17.099794 18178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.105820 18178 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:45473
I20260808 00:38:17.105881 18405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:45473 every 8 connection(s)
I20260808 00:38:17.106195 18178 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:38:17.111107 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.106890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:42339 (local address 127.1.208.62:43721)
0808 00:38:17.106954 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.106956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.107181 (+ 225us) server_negotiation.cc:407] Connection header received
0808 00:38:17.107676 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.107678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.107689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.107740 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.108394 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.108890 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.109921 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.110111 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.110165 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:17.110906 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.110907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.110908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.110912 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.110924 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.110928 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.110961 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.111024 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.111045 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:38:17.111299 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.106873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43721 (local address 127.1.208.2:42339)
0808 00:38:17.107059 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:17.107093 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:17.107589 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:17.107794 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:17.107798 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:17.107806 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:17.108289 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:17.108293 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.108960 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.108961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:17.109839 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:17.109843 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.110171 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.110172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:17.110278 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:17.110777 (+ 499us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:17.110778 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:17.110806 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:17.110828 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:17.110969 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:17.110971 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:17.110973 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:38:17.111014 (+ 41us) client_negotiation.cc:240] Negotiation successful
0808 00:38:17.111093 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260808 00:38:17.111757 18406 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43721
I20260808 00:38:17.111857 18406 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:17.112008 18406 heartbeater.cc:511] Master 127.1.208.62:43721 requested a full tablet report, sending...
I20260808 00:38:17.112382 17843 ts_manager.cc:194] Registered new tserver with Master: 8c89ac6b7e7541608888df941f0d05ce (127.1.208.2:45473)
I20260808 00:38:17.112710 17843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.2:42339
I20260808 00:38:17.113988 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 18178
I20260808 00:38:17.114066 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:38:17.114210 1856 external_mini_cluster.cc:1428] Setting key ca23b5424dd17c1598e8ae07db54d2ee
I20260808 00:38:17.115020 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--builtin_ntp_servers=127.1.208.20:46287
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 00:38:17.196508 18413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.196698 18413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.196717 18413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.197815 18413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.197860 18413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.197881 18413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:38:17.199138 18413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46287
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:43721
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.199339 18413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.199608 18413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.204444 18425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.204490 18424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.204456 18428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.204536 18413 server_base.cc:1034] running on GCE node
I20260808 00:38:17.204818 18413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.205015 18413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.206148 18413 hybrid_clock.cc:648] HybridClock initialized: now 1786149497206136 us; error 29 us; skew 500 ppm
I20260808 00:38:17.207297 18413 webserver.cc:460] Webserver started at http://127.1.208.3:46481/ using document root <none> and password file <none>
I20260808 00:38:17.207541 18413 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.207584 18413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.207711 18413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.208725 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "ec7338025fed4cf5bde74f265872dec8"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2e4c85c188d8bd2c263b151572f695ef"
server_key_iv: "a5a45e438f7736d275077b08b3c375fc"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.209153 18413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "ec7338025fed4cf5bde74f265872dec8"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2e4c85c188d8bd2c263b151572f695ef"
server_key_iv: "a5a45e438f7736d275077b08b3c375fc"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.210521 18413 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:38:17.211174 18443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.211321 18413 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:38:17.211426 18413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "ec7338025fed4cf5bde74f265872dec8"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2e4c85c188d8bd2c263b151572f695ef"
server_key_iv: "a5a45e438f7736d275077b08b3c375fc"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.211493 18413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.275486 18413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.275837 18413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.275995 18413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.276250 18413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:17.276633 18413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:17.276670 18413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.276695 18413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:17.276716 18413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.282653 18413 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:39419
I20260808 00:38:17.282701 18632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:39419 every 8 connection(s)
I20260808 00:38:17.283004 18413 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:38:17.287882 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.283749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:60623 (local address 127.1.208.62:43721)
0808 00:38:17.283816 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.283818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.284049 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:38:17.284539 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.284540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.284550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.284600 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.285266 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.285791 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.286743 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.286925 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.286969 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:17.287670 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.287671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.287671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.287676 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.287691 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.287695 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.287733 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.287794 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.287820 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:38:17.288067 18635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.283725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:43721 (local address 127.1.208.3:60623)
0808 00:38:17.283931 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:17.283967 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:17.284457 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:17.284670 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:17.284673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:17.284682 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:17.285198 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:17.285202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.285893 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.285894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:17.286679 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:17.286681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:17.286980 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:17.286980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:17.287088 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:17.287543 (+ 455us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:17.287544 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:17.287572 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:17.287595 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:17.287739 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:17.287741 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:17.287743 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:38:17.287789 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:38:17.287874 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260808 00:38:17.288498 18633 heartbeater.cc:347] Connected to a master server at 127.1.208.62:43721
I20260808 00:38:17.288589 18633 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:17.288764 18633 heartbeater.cc:511] Master 127.1.208.62:43721 requested a full tablet report, sending...
I20260808 00:38:17.289088 17843 ts_manager.cc:194] Registered new tserver with Master: ec7338025fed4cf5bde74f265872dec8 (127.1.208.3:39419)
I20260808 00:38:17.289487 17843 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.3:60623
I20260808 00:38:17.290874 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 18413
I20260808 00:38:17.290954 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:38:17.291129 1856 external_mini_cluster.cc:1428] Setting key 0466afeba2f297060c113f3f58dcbfc5
I20260808 00:38:17.291863 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:38:17.293623 17939 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:17.293157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.1.208.62:43721)
0808 00:38:17.293202 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.293203 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.293310 (+ 107us) server_negotiation.cc:407] Connection header received
0808 00:38:17.293383 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.293384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.293394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.293442 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.293559 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52840: BlockingRecv error: recv got EOF from 127.0.0.1:52840 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:38:17.293653 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 17951
I20260808 00:38:17.315135 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 18178
I20260808 00:38:17.324143 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 18413
I20260808 00:38:17.362314 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 17777
2026-08-08T00:38:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (692 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T00:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:38:17Z Disabled control of system clock
I20260808 00:38:17.393918 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38831
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:38:17.475651 18648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.475868 18648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.475889 18648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.477087 18648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:38:17.477128 18648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.477139 18648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.477150 18648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:38:17.477166 18648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:38:17.478684 18648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38831
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.478912 18648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.479168 18648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.483850 18663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.483852 18658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.483879 18648 server_base.cc:1034] running on GCE node
W20260808 00:38:17.483964 18660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.484364 18648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.484618 18648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.485764 18648 hybrid_clock.cc:648] HybridClock initialized: now 1786149497485751 us; error 28 us; skew 500 ppm
I20260808 00:38:17.486960 18648 webserver.cc:460] Webserver started at http://127.1.208.62:40023/ using document root <none> and password file <none>
I20260808 00:38:17.487191 18648 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.487237 18648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.487378 18648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.488452 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "7502be2a025841c2b6b838b17b85cd43"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "3fecc3bf257c0feeae7bc528b41d7fa5"
server_key_iv: "17fb8fabbf2202e87cdff940be06135f"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.488752 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "7502be2a025841c2b6b838b17b85cd43"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "3fecc3bf257c0feeae7bc528b41d7fa5"
server_key_iv: "17fb8fabbf2202e87cdff940be06135f"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.490031 18648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:38:17.490712 18671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.490859 18648 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:17.490926 18648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "7502be2a025841c2b6b838b17b85cd43"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "3fecc3bf257c0feeae7bc528b41d7fa5"
server_key_iv: "17fb8fabbf2202e87cdff940be06135f"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.490995 18648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.494318 18648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.494573 18648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.494690 18648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.498775 18648 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38831
I20260808 00:38:17.498804 18768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38831 every 8 connection(s)
I20260808 00:38:17.499111 18648 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:38:17.500028 18769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:17.502440 18769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43: Bootstrap starting.
I20260808 00:38:17.503090 18769 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:17.503383 18769 log.cc:824] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:17.504068 18769 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43: No bootstrap required, opened a new log
I20260808 00:38:17.505975 18769 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } }
I20260808 00:38:17.506125 18769 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:17.506148 18769 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7502be2a025841c2b6b838b17b85cd43, State: Initialized, Role: FOLLOWER
I20260808 00:38:17.506304 18769 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } }
I20260808 00:38:17.506368 18769 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:38:17.506390 18769 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:38:17.506443 18769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:17.506970 18769 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } }
I20260808 00:38:17.507064 18769 leader_election.cc:302] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7502be2a025841c2b6b838b17b85cd43; no voters:
I20260808 00:38:17.507220 18769 leader_election.cc:288] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:38:17.507268 18772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:38:17.507419 18772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [term 1 LEADER]: Becoming Leader. State: Replica: 7502be2a025841c2b6b838b17b85cd43, State: Running, Role: LEADER
I20260808 00:38:17.507485 18769 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:38:17.507512 18772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } }
I20260808 00:38:17.507896 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 18648
I20260808 00:38:17.507997 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:38:17.508050 18774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7502be2a025841c2b6b838b17b85cd43. Latest consensus state: current_term: 1 leader_uuid: "7502be2a025841c2b6b838b17b85cd43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } } }
I20260808 00:38:17.508119 18774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:17.508193 1856 external_mini_cluster.cc:1428] Setting key 15c6e9950f5625c48451ef029e37558f
I20260808 00:38:17.508280 18773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7502be2a025841c2b6b838b17b85cd43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7502be2a025841c2b6b838b17b85cd43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38831 } } }
I20260808 00:38:17.508633 18773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:17.508653 18790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:38:17.509218 18790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:38:17.510852 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.508876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.1.208.62:38831)
0808 00:38:17.509260 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.509269 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.509325 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:38:17.509446 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.509453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.509498 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.509632 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.510287 (+ 655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.510292 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.510292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.510297 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.510361 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.510376 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.510437 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.510603 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.510660 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":60,"threads_started":1}
I20260808 00:38:17.511402 18790 catalog_manager.cc:1347] Generated new cluster ID: 4ecb54cea090408db71f0bbee1d8e832
I20260808 00:38:17.511461 18790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:38:17.538920 18790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:38:17.539119 18790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:38:17.544855 18790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7502be2a025841c2b6b838b17b85cd43: Generated new TSK 0
I20260808 00:38:17.545081 18790 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:38:17.548197 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--builtin_ntp_servers=127.1.208.20:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:38:17.637336 18810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.637578 18810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.637614 18810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.638873 18810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.638913 18810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.638950 18810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:38:17.640394 18810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.640650 18810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.640949 18810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.646031 18820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.646062 18826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.646031 18823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.646267 18810 server_base.cc:1034] running on GCE node
I20260808 00:38:17.646483 18810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.646723 18810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.647862 18810 hybrid_clock.cc:648] HybridClock initialized: now 1786149497647831 us; error 48 us; skew 500 ppm
I20260808 00:38:17.649124 18810 webserver.cc:460] Webserver started at http://127.1.208.1:37111/ using document root <none> and password file <none>
I20260808 00:38:17.649410 18810 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.649466 18810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.649601 18810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.650684 18810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "438b8aa0cdf048178c7fb4dddc76ec2f"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "48e616d088f19456421426b344d3edcf"
server_key_iv: "0b4cb79860285c52b58e0b9c036f6d4a"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.651034 18810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "438b8aa0cdf048178c7fb4dddc76ec2f"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "48e616d088f19456421426b344d3edcf"
server_key_iv: "0b4cb79860285c52b58e0b9c036f6d4a"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.652642 18810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:38:17.653542 18836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.653821 18810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:38:17.653941 18810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "438b8aa0cdf048178c7fb4dddc76ec2f"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "48e616d088f19456421426b344d3edcf"
server_key_iv: "0b4cb79860285c52b58e0b9c036f6d4a"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.654013 18810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.657498 18810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.657829 18810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.657970 18810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.658169 18810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:17.658478 18810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:17.658511 18810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.658540 18810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:17.658565 18810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.664376 18810 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35909
I20260808 00:38:17.664438 19030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35909 every 8 connection(s)
I20260808 00:38:17.664757 18810 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:38:17.667691 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.665544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:50835 (local address 127.1.208.62:38831)
0808 00:38:17.665625 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.665628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.665822 (+ 194us) server_negotiation.cc:407] Connection header received
0808 00:38:17.666317 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.666318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.666326 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.666365 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.667500 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.667501 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.667502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.667506 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.667522 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.667526 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.667572 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.667622 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.667635 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 00:38:17.667898 19036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.665504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38831 (local address 127.1.208.1:50835)
0808 00:38:17.665697 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:17.665741 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:17.666218 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:17.666425 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:17.666430 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:17.666439 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:17.667343 (+ 904us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:17.667343 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:17.667374 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:17.667406 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:17.667577 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:17.667579 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:17.667582 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:38:17.667630 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:38:17.667681 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":77,"threads_started":1}
I20260808 00:38:17.668372 19032 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38831
I20260808 00:38:17.668462 19032 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:17.668589 19032 heartbeater.cc:511] Master 127.1.208.62:38831 requested a full tablet report, sending...
I20260808 00:38:17.669085 18697 ts_manager.cc:194] Registered new tserver with Master: 438b8aa0cdf048178c7fb4dddc76ec2f (127.1.208.1:35909)
I20260808 00:38:17.674259 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 18810
I20260808 00:38:17.674373 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:38:17.674559 1856 external_mini_cluster.cc:1428] Setting key 62cc3cfaa2dbbe7c683e0c996ef9c7e5
I20260808 00:38:17.675616 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--builtin_ntp_servers=127.1.208.20:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:38:17.760890 19037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.761125 19037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.761160 19037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.762440 19037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.762488 19037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.762526 19037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:38:17.763934 19037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.764189 19037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.764530 19037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.769532 19047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.769563 19044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.769575 19045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.769605 19037 server_base.cc:1034] running on GCE node
I20260808 00:38:17.770015 19037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.770265 19037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.771425 19037 hybrid_clock.cc:648] HybridClock initialized: now 1786149497771394 us; error 33 us; skew 500 ppm
I20260808 00:38:17.772651 19037 webserver.cc:460] Webserver started at http://127.1.208.2:40781/ using document root <none> and password file <none>
I20260808 00:38:17.772883 19037 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.772931 19037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.773063 19037 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.774207 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "6d1d76b8abc7419eb88eb73066d86a73"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "4efe9f830197eb3f3d3c8c64af250110"
server_key_iv: "610ecc7088aa26dcf28e0f35994a03e0"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.774582 19037 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "6d1d76b8abc7419eb88eb73066d86a73"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "4efe9f830197eb3f3d3c8c64af250110"
server_key_iv: "610ecc7088aa26dcf28e0f35994a03e0"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.775857 19037 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:38:17.776611 19060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.776763 19037 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:17.776851 19037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "6d1d76b8abc7419eb88eb73066d86a73"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "4efe9f830197eb3f3d3c8c64af250110"
server_key_iv: "610ecc7088aa26dcf28e0f35994a03e0"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.776918 19037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.780292 19037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.780552 19037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.780673 19037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.780872 19037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:17.781176 19037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:17.781210 19037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.781239 19037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:17.781265 19037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.787024 19037 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:33481
I20260808 00:38:17.787088 19247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:33481 every 8 connection(s)
I20260808 00:38:17.787395 19037 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:38:17.790284 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.788134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:56799 (local address 127.1.208.62:38831)
0808 00:38:17.788194 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.788196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.788435 (+ 239us) server_negotiation.cc:407] Connection header received
0808 00:38:17.788921 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.788922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.788928 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.788988 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.790092 (+ 1104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.790093 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.790093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.790097 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.790116 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.790120 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.790170 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.790219 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.790231 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:38:17.790437 19256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.788113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38831 (local address 127.1.208.2:56799)
0808 00:38:17.788307 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:17.788348 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:17.788837 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:17.789043 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:17.789047 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:17.789056 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:17.789943 (+ 887us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:17.789943 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:17.789973 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:17.790002 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:17.790167 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:17.790169 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:17.790171 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:38:17.790220 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:38:17.790267 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":83,"threads_started":1}
I20260808 00:38:17.790887 19251 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38831
I20260808 00:38:17.790975 19251 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:17.791134 19251 heartbeater.cc:511] Master 127.1.208.62:38831 requested a full tablet report, sending...
I20260808 00:38:17.791204 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 19037
I20260808 00:38:17.791301 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:38:17.791504 1856 external_mini_cluster.cc:1428] Setting key 64d4b5a92bbdc1151716a64e850f2b3a
I20260808 00:38:17.791569 18697 ts_manager.cc:194] Registered new tserver with Master: 6d1d76b8abc7419eb88eb73066d86a73 (127.1.208.2:33481)
I20260808 00:38:17.792347 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--builtin_ntp_servers=127.1.208.20:41523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 00:38:17.875481 19257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:17.875697 19257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:17.875728 19257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:17.876881 19257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:17.876919 19257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:17.876953 19257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:38:17.878300 19257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:41523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:17.878559 19257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:17.878886 19257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:17.884919 19274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.884935 19278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:17.885187 19276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:17.885320 19257 server_base.cc:1034] running on GCE node
I20260808 00:38:17.885581 19257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:17.885828 19257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:17.886989 19257 hybrid_clock.cc:648] HybridClock initialized: now 1786149497886973 us; error 29 us; skew 500 ppm
I20260808 00:38:17.888407 19257 webserver.cc:460] Webserver started at http://127.1.208.3:41387/ using document root <none> and password file <none>
I20260808 00:38:17.888598 19257 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:17.888636 19257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:17.888731 19257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:17.889846 19257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "22f5fb756a0b42b39d9d999640d4fedc"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2f1b5bde5374d83ee2c61cab1d2a1f7b"
server_key_iv: "f801e3aaa45012cb78b0dec9b5f0dc5b"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.890175 19257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "22f5fb756a0b42b39d9d999640d4fedc"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2f1b5bde5374d83ee2c61cab1d2a1f7b"
server_key_iv: "f801e3aaa45012cb78b0dec9b5f0dc5b"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.891342 19257 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:38:17.891965 19290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.892125 19257 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:17.892210 19257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "22f5fb756a0b42b39d9d999640d4fedc"
format_stamp: "Formatted at 2026-08-08 00:38:17 on dist-test-slave-gjw7"
server_key: "2f1b5bde5374d83ee2c61cab1d2a1f7b"
server_key_iv: "f801e3aaa45012cb78b0dec9b5f0dc5b"
server_key_version: "encryptionkey@0"
I20260808 00:38:17.892278 19257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:17.895690 19257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:17.895959 19257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:17.896080 19257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:17.896284 19257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:17.896589 19257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:17.896622 19257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.896649 19257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:17.896667 19257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:17.902413 19257 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:46241
I20260808 00:38:17.902454 19483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:46241 every 8 connection(s)
I20260808 00:38:17.902766 19257 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:38:17.905603 18804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.903500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:57805 (local address 127.1.208.62:38831)
0808 00:38:17.903590 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.903592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.903765 (+ 173us) server_negotiation.cc:407] Connection header received
0808 00:38:17.904233 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.904234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.904239 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.904278 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.905412 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:17.905414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:17.905414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:38:17.905418 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:17.905433 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:38:17.905437 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:17.905476 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:17.905537 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:38:17.905548 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:38:17.905787 19489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:17.903469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38831 (local address 127.1.208.3:57805)
0808 00:38:17.903651 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:17.903686 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:17.904141 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:17.904336 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:17.904339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:17.904348 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:17.905211 (+ 863us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:38:17.905212 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:17.905241 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:38:17.905271 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:17.905483 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:17.905487 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:17.905490 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:38:17.905548 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:38:17.905599 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260808 00:38:17.906235 19484 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38831
I20260808 00:38:17.906316 19484 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:17.906479 19484 heartbeater.cc:511] Master 127.1.208.62:38831 requested a full tablet report, sending...
I20260808 00:38:17.906845 18697 ts_manager.cc:194] Registered new tserver with Master: 22f5fb756a0b42b39d9d999640d4fedc (127.1.208.3:46241)
I20260808 00:38:17.907866 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 19257
I20260808 00:38:17.907967 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:38:17.908154 1856 external_mini_cluster.cc:1428] Setting key 053171f4795ef214c8ec368137003551
I20260808 00:38:17.908972 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 00:38:17.910887 18804 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:38:17.910372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48076 (local address 127.1.208.62:38831)
0808 00:38:17.910447 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:17.910449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:17.910567 (+ 118us) server_negotiation.cc:407] Connection header received
0808 00:38:17.910631 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:17.910632 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:17.910638 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:17.910680 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:17.910819 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48076: BlockingRecv error: recv got EOF from 127.0.0.1:48076 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 00:38:17.911172 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 18810
I20260808 00:38:17.932343 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 19037
I20260808 00:38:17.957233 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 19257
I20260808 00:38:17.967849 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 18648
2026-08-08T00:38:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (609 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:38:18 dist-test-slave-gjw7 krb5kdc[19503](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:38:18 dist-test-slave-gjw7 krb5kdc[19503](info): set up 2 sockets
Aug 08 00:38:18 dist-test-slave-gjw7 krb5kdc[19503](info): commencing operation
krb5kdc: starting...
W20260808 00:38:19.144637 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:38:19.262056 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46715
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:46715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:19.344096 19525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:19.344288 19525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:19.344309 19525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:19.345489 19525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:38:19.345530 19525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:19.345543 19525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:19.345567 19525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:38:19.345583 19525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:38:19.347101 19525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:46715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:19.347330 19525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:19.347540 19525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:19.352094 19540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:19.352170 19536 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.352221 19525 server_base.cc:1034] running on GCE node
W20260808 00:38:19.352221 19537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.352633 19525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:19.352903 19525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:19.354043 19525 hybrid_clock.cc:648] HybridClock initialized: now 1786149499354022 us; error 39 us; skew 500 ppm
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:19.356405 19525 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:38:19.356806 19525 webserver.cc:460] Webserver started at http://127.1.208.62:41457/ using document root <none> and password file <none>
I20260808 00:38:19.357031 19525 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:19.357080 19525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:19.357210 19525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:19.358258 19525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "d8b457430029496cadedcd70049a83c1"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "fe87546c85e3ad73b820d06dfc2a4e85"
server_key_iv: "0f289acbca6e966db882cf291781ac79"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.358603 19525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "d8b457430029496cadedcd70049a83c1"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "fe87546c85e3ad73b820d06dfc2a4e85"
server_key_iv: "0f289acbca6e966db882cf291781ac79"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.359838 19525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:38:19.360455 19552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.360610 19525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:38:19.360685 19525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "d8b457430029496cadedcd70049a83c1"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "fe87546c85e3ad73b820d06dfc2a4e85"
server_key_iv: "0f289acbca6e966db882cf291781ac79"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.360750 19525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:19.371414 19525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:19.373448 19525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:19.373585 19525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:19.377430 19525 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:46715
I20260808 00:38:19.377465 19631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:46715 every 8 connection(s)
I20260808 00:38:19.377775 19525 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:38:19.378747 19632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:19.380954 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: Bootstrap starting.
I20260808 00:38:19.381551 19632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:19.381783 19632 log.cc:824] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:19.382426 19632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: No bootstrap required, opened a new log
I20260808 00:38:19.384001 19632 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } }
I20260808 00:38:19.384136 19632 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:19.384158 19632 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8b457430029496cadedcd70049a83c1, State: Initialized, Role: FOLLOWER
I20260808 00:38:19.384271 19632 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } }
I20260808 00:38:19.384347 19632 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:38:19.384379 19632 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:38:19.384419 19632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:19.385149 19632 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } }
I20260808 00:38:19.385237 19632 leader_election.cc:302] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8b457430029496cadedcd70049a83c1; no voters:
I20260808 00:38:19.385396 19632 leader_election.cc:288] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:38:19.385435 19642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:38:19.385571 19642 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [term 1 LEADER]: Becoming Leader. State: Replica: d8b457430029496cadedcd70049a83c1, State: Running, Role: LEADER
I20260808 00:38:19.385635 19632 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:38:19.385654 19642 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } }
I20260808 00:38:19.386059 19644 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8b457430029496cadedcd70049a83c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } } }
I20260808 00:38:19.386049 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8b457430029496cadedcd70049a83c1. Latest consensus state: current_term: 1 leader_uuid: "d8b457430029496cadedcd70049a83c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8b457430029496cadedcd70049a83c1" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46715 } } }
I20260808 00:38:19.386140 19644 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:19.386147 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1 [sys.catalog]: This master's current role is: LEADER
I20260808 00:38:19.386708 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 19525
I20260808 00:38:19.386709 19659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:38:19.386811 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:38:19.387003 1856 external_mini_cluster.cc:1428] Setting key d4ad7e46afc98759920afa47d60064af
I20260808 00:38:19.387214 19659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:38:19.388790 19659 catalog_manager.cc:1347] Generated new cluster ID: cc9d800f93ec4628a13fc0023e77dbf1
I20260808 00:38:19.388844 19659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:19.397037 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.387460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56164 (local address 127.1.208.62:46715)
0808 00:38:19.388107 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.388116 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.388146 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:38:19.388257 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.388264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.388329 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.388461 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:19.389180 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.390153 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.390892 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.391302 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.391405 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.393616 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:19.393620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:19.393620 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:19.393633 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:19.395615 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.395941 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.395944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.395944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.396043 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.396222 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.396223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.396223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.396451 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:19.396615 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.396684 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.396817 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":76,"threads_started":1}
I20260808 00:38:19.411725 19659 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:38:19.412456 19659 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:38:19.416309 19659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: Generated new TSK 0
I20260808 00:38:19.416487 19659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:38:19.444139 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:19.525050 19685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:19.525234 19685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:19.525256 19685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:19.526365 19685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:19.526405 19685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:19.526432 19685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:38:19.527829 19685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:19.528039 19685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:19.528250 19685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:19.532924 19693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:19.532953 19692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:19.533005 19699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.533209 19685 server_base.cc:1034] running on GCE node
I20260808 00:38:19.533402 19685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:19.533605 19685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:19.534734 19685 hybrid_clock.cc:648] HybridClock initialized: now 1786149499534720 us; error 30 us; skew 500 ppm
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:19.541081 19685 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:38:19.541507 19685 webserver.cc:460] Webserver started at http://127.1.208.1:33237/ using document root <none> and password file <none>
I20260808 00:38:19.541682 19685 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:19.541719 19685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:19.541808 19685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:19.542761 19685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "259c443bdee848659f5c49f58b2f3d3a"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "e9119a7111c11b05250511aebe811df5"
server_key_iv: "d28a0035f0e6e615ee913f699b5c8ae2"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.543062 19685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "259c443bdee848659f5c49f58b2f3d3a"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "e9119a7111c11b05250511aebe811df5"
server_key_iv: "d28a0035f0e6e615ee913f699b5c8ae2"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.544215 19685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:38:19.544847 19710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.545035 19685 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.545120 19685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "259c443bdee848659f5c49f58b2f3d3a"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "e9119a7111c11b05250511aebe811df5"
server_key_iv: "d28a0035f0e6e615ee913f699b5c8ae2"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.545176 19685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:19.554131 19685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:19.555764 19685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:19.555873 19685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:19.556099 19685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:19.556382 19685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:19.556428 19685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.556447 19685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:19.556469 19685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.562104 19685 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35177
I20260808 00:38:19.562172 19899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35177 every 8 connection(s)
I20260808 00:38:19.562448 19685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:19.569061 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 19685
I20260808 00:38:19.569156 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:38:19.569396 1856 external_mini_cluster.cc:1428] Setting key c33bb05b3beb312f0f2f3b8494ab37df
I20260808 00:38:19.571623 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.563108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:44623 (local address 127.1.208.62:46715)
0808 00:38:19.563153 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.563155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.563353 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:38:19.563868 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.563869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.563886 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.563938 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:19.564907 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.565457 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.566425 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.566639 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.566695 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.568733 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:19.568735 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:19.568736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:19.568744 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:19.570293 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.570710 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.570711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.570711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.570760 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.571005 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.571006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.571006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.571098 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:19.571146 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.571472 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.571556 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:38:19.571797 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.563086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46715 (local address 127.1.208.1:44623)
0808 00:38:19.563234 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.563273 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.563762 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.564018 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.564023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.564622 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:19.564806 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.564811 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.565541 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.565542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.566335 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.566337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.566713 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.566714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.566835 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.567296 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:19.567305 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:19.568637 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:19.570367 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.570375 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.570378 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.570643 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.570818 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.570818 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.570819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.570954 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.571147 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:19.571153 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:19.571303 (+ 150us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.571410 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.571570 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
I20260808 00:38:19.572417 19901 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46715
I20260808 00:38:19.572500 19901 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:19.572670 19901 heartbeater.cc:511] Master 127.1.208.62:46715 requested a full tablet report, sending...
I20260808 00:38:19.573215 19580 ts_manager.cc:194] Registered new tserver with Master: 259c443bdee848659f5c49f58b2f3d3a (127.1.208.1:35177)
I20260808 00:38:19.574040 19580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:44623
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:38:19.595304 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:19.678272 19908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:19.678479 19908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:19.678501 19908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:19.679625 19908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:19.679667 19908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:19.679693 19908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:38:19.681116 19908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:19.681366 19908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:19.681591 19908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:19.686515 19916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:19.686523 19915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.686589 19908 server_base.cc:1034] running on GCE node
W20260808 00:38:19.686574 19918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.686919 19908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:19.687125 19908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:19.688257 19908 hybrid_clock.cc:648] HybridClock initialized: now 1786149499688247 us; error 31 us; skew 500 ppm
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:19.690742 19908 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:38:19.691236 19908 webserver.cc:460] Webserver started at http://127.1.208.2:36725/ using document root <none> and password file <none>
I20260808 00:38:19.691421 19908 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:19.691461 19908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:19.691561 19908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:19.692535 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "16a1cb3d536846bcb6aa4447f9e4ad03"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "5b27a835fbc074ec98b2f11f9b571652"
server_key_iv: "63bfe02e6d5dc8996e9b3cf88e329e52"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.692834 19908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "16a1cb3d536846bcb6aa4447f9e4ad03"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "5b27a835fbc074ec98b2f11f9b571652"
server_key_iv: "63bfe02e6d5dc8996e9b3cf88e329e52"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.694067 19908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:38:19.694715 19929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.694859 19908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:38:19.694938 19908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "16a1cb3d536846bcb6aa4447f9e4ad03"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "5b27a835fbc074ec98b2f11f9b571652"
server_key_iv: "63bfe02e6d5dc8996e9b3cf88e329e52"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.695005 19908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:19.725383 19908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:19.727577 19908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:19.727711 19908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:19.727941 19908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:19.728263 19908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:19.728298 19908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.728329 19908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:19.728348 19908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.734210 19908 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:35207
I20260808 00:38:19.734278 20127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:35207 every 8 connection(s)
I20260808 00:38:19.734586 19908 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:19.740367 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 19908
I20260808 00:38:19.740463 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:38:19.740664 1856 external_mini_cluster.cc:1428] Setting key 710d821fd1ea5ec6b298db35b17d3c78
I20260808 00:38:19.743510 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.735250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:50133 (local address 127.1.208.62:46715)
0808 00:38:19.735308 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.735311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.735510 (+ 199us) server_negotiation.cc:407] Connection header received
0808 00:38:19.735999 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.736000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.736010 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.736059 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:19.736944 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.737486 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.738403 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.738586 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.738629 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.740807 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:19.740809 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:19.740809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:19.740816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:19.742151 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.742597 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.742598 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.742598 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.742649 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.742885 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.742885 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.742885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.743032 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:19.743098 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.743335 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.743422 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:38:19.743602 20134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.735228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46715 (local address 127.1.208.2:50133)
0808 00:38:19.735382 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.735426 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.735905 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.736117 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.736121 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.736679 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:19.736874 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.736879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.737560 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.737561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.738342 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.738344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.738631 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.738632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.738722 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.739187 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:19.739195 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:19.740667 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:19.742235 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.742242 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.742245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.742527 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.742699 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.742699 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.742700 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.742823 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.743096 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:19.743101 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:19.743219 (+ 118us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.743302 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.743429 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260808 00:38:19.744055 20128 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46715
I20260808 00:38:19.744150 20128 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:19.744333 20128 heartbeater.cc:511] Master 127.1.208.62:46715 requested a full tablet report, sending...
I20260808 00:38:19.744699 19580 ts_manager.cc:194] Registered new tserver with Master: 16a1cb3d536846bcb6aa4447f9e4ad03 (127.1.208.2:35207)
I20260808 00:38:19.745056 19580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:50133
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:38:19.767570 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:39483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:38:19.849712 20139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:38:19.849916 20139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:38:19.849937 20139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:38:19.850996 20139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:38:19.851027 20139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:38:19.851051 20139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:38:19.852438 20139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:39483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:46715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:38:19.852629 20139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:38:19.852802 20139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:38:19.857625 20153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.857761 20139 server_base.cc:1034] running on GCE node
W20260808 00:38:19.857618 20156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:38:19.857900 20154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:38:19.858131 20139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:38:19.858376 20139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:38:19.859513 20139 hybrid_clock.cc:648] HybridClock initialized: now 1786149499859506 us; error 27 us; skew 500 ppm
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:38:19.861835 20139 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:38:19.862264 20139 webserver.cc:460] Webserver started at http://127.1.208.3:38389/ using document root <none> and password file <none>
I20260808 00:38:19.862483 20139 fs_manager.cc:359] Metadata directory not provided
I20260808 00:38:19.862532 20139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:38:19.862658 20139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:38:19.863756 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "2912822bec8a42439a3df305dff78f21"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "6dc77ea6a35feb17882c7d65d63439e1"
server_key_iv: "056be38f90a6a9e6622d6f4a5d146292"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.864140 20139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "2912822bec8a42439a3df305dff78f21"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "6dc77ea6a35feb17882c7d65d63439e1"
server_key_iv: "056be38f90a6a9e6622d6f4a5d146292"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.865604 20139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:38:19.866401 20171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.866590 20139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:38:19.866674 20139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "2912822bec8a42439a3df305dff78f21"
format_stamp: "Formatted at 2026-08-08 00:38:19 on dist-test-slave-gjw7"
server_key: "6dc77ea6a35feb17882c7d65d63439e1"
server_key_iv: "056be38f90a6a9e6622d6f4a5d146292"
server_key_version: "encryptionkey@0"
I20260808 00:38:19.866744 20139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:38:19.888290 20139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:38:19.890167 20139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:38:19.890321 20139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:38:19.890532 20139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:38:19.890838 20139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:38:19.890872 20139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.890902 20139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:38:19.890928 20139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:38:19.896854 20139 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:36967
I20260808 00:38:19.896909 20366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:36967 every 8 connection(s)
I20260808 00:38:19.897224 20139 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:38:19.902629 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 20139
I20260808 00:38:19.902721 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:38:19.902906 1856 external_mini_cluster.cc:1428] Setting key 47ed548c8975c13da206574ffc1e13cb
Aug 08 00:38:19 dist-test-slave-gjw7 krb5kdc[19503](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149499, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:38:19.905802 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.897937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:53209 (local address 127.1.208.62:46715)
0808 00:38:19.898016 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.898018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.898218 (+ 200us) server_negotiation.cc:407] Connection header received
0808 00:38:19.898767 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.898768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.898780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.898828 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:19.899714 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.900211 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.901130 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.901354 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.901404 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.903542 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:19.903543 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:19.903543 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:19.903550 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:19.904721 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.905056 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.905056 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.905057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.905103 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.905266 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.905267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.905267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.905414 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:19.905465 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.905678 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.905739 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:38:19.905993 20371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.897928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:46715 (local address 127.1.208.3:53209)
0808 00:38:19.898101 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.898139 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.898625 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.898888 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.898892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.899454 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:38:19.899646 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.899651 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.900297 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.900297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.901064 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.901066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.901422 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.901423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.901534 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.901924 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:38:19.901932 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:38:19.903454 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:38:19.904792 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.904798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.904800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.905001 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.905138 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:38:19.905138 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:38:19.905139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:38:19.905227 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:38:19.905464 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:38:19.905467 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:38:19.905579 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.905659 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.905774 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":92,"threads_started":1}
I20260808 00:38:19.906445 20369 heartbeater.cc:347] Connected to a master server at 127.1.208.62:46715
I20260808 00:38:19.906535 20369 heartbeater.cc:464] Registering TS with master...
I20260808 00:38:19.906682 20369 heartbeater.cc:511] Master 127.1.208.62:46715 requested a full tablet report, sending...
I20260808 00:38:19.907025 19580 ts_manager.cc:194] Registered new tserver with Master: 2912822bec8a42439a3df305dff78f21 (127.1.208.3:36967)
I20260808 00:38:19.907410 19580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:53209
I20260808 00:38:19.914631 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:38:19.921908 19677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.916070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56466 (local address 127.1.208.62:46715)
0808 00:38:19.916131 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.916133 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.916274 (+ 141us) server_negotiation.cc:407] Connection header received
0808 00:38:19.916365 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.916366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.916377 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.916435 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:38:19.917037 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.917566 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.918300 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.918486 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.918548 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.919701 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:38:19.919702 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:38:19.919702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:38:19.919710 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:38:19.920745 (+ 1035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.921072 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.921073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.921073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.921132 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:38:19.921349 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:38:19.921350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:38:19.921350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:38:19.921503 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:38:19.921603 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.921734 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.921816 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:38:19.923338 19580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:38:19.924938 19580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:38:19.931222 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35207 (local address 127.0.0.1:42340)
0808 00:38:19.927808 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.927818 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.927905 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.928141 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.928142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.928144 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:38:19.928193 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.928195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.929415 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.929416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.930294 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.930297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.930933 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.930934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.931012 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.931027 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.931089 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.931142 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":107,"threads_started":1}
I20260808 00:38:19.931238 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.1.208.2:35207)
0808 00:38:19.927668 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.927673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.927875 (+ 202us) server_negotiation.cc:407] Connection header received
0808 00:38:19.927976 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.927979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.927993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.928060 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:38:19.928275 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.929324 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.930365 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.930855 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.930917 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.930980 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.931111 (+ 131us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.931154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260808 00:38:19.931524 20391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36967 (local address 127.0.0.1:56076)
0808 00:38:19.927385 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.927425 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.927491 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.928163 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.928164 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.928166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:38:19.928210 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.928212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.929009 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.929010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.930340 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.930341 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.931242 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.931244 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.931364 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.931373 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.931402 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.931441 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":1,"thread_start_us":43,"threads_started":1}
I20260808 00:38:19.931524 20389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35177 (local address 127.0.0.1:37156)
0808 00:38:19.927385 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.927425 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.927491 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.927850 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.927854 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.927856 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:38:19.927926 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.927928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.929187 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.929188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.930314 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.930316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.931310 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.931311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.931364 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.931371 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.931418 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.931450 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":10,"thread_start_us":83,"threads_started":1}
I20260808 00:38:19.931808 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56076 (local address 127.1.208.3:36967)
0808 00:38:19.927776 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.927779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.927787 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:38:19.927823 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.927826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.927840 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.928187 (+ 347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:38:19.928269 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.928917 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.930426 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.931135 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.931570 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.931640 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.931668 (+ 28us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.931710 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260808 00:38:19.931856 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.927431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37156 (local address 127.1.208.1:35177)
0808 00:38:19.927666 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.927671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.927682 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:38:19.927725 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.927728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.927743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.927842 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:38:19.927986 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.929085 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.930425 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.931135 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.931203 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.931279 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.931739 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.931781 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":86,"threads_started":1}
I20260808 00:38:19.932017 20017 tablet_service.cc:1467] Processing CreateTablet for tablet 24315bb8065846d6b736f51b84719391 (DEFAULT_TABLE table=test-table [id=b034eb8837b2476497a7000ccf412f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:38:19.932406 20017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24315bb8065846d6b736f51b84719391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:19.932456 19791 tablet_service.cc:1467] Processing CreateTablet for tablet 24315bb8065846d6b736f51b84719391 (DEFAULT_TABLE table=test-table [id=b034eb8837b2476497a7000ccf412f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:38:19.932811 19791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24315bb8065846d6b736f51b84719391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:19.934051 20257 tablet_service.cc:1467] Processing CreateTablet for tablet 24315bb8065846d6b736f51b84719391 (DEFAULT_TABLE table=test-table [id=b034eb8837b2476497a7000ccf412f45]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:38:19.934486 20257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24315bb8065846d6b736f51b84719391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:38:19.935464 20395 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Bootstrap starting.
I20260808 00:38:19.935937 20397 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: Bootstrap starting.
I20260808 00:38:19.936123 20395 tablet_bootstrap.cc:654] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:19.936668 20397 tablet_bootstrap.cc:654] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:19.936682 20395 log.cc:824] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:19.937170 20397 log.cc:824] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:19.937485 20403 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Bootstrap starting.
I20260808 00:38:19.937530 20395 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: No bootstrap required, opened a new log
I20260808 00:38:19.937593 20395 ts_tablet_manager.cc:1397] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:38:19.938076 20403 tablet_bootstrap.cc:654] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Neither blocks nor log segments found. Creating new log.
I20260808 00:38:19.938513 20403 log.cc:824] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Log is configured to *not* fsync() on all Append() calls
I20260808 00:38:19.939165 20395 raft_consensus.cc:348] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.939280 20403 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: No bootstrap required, opened a new log
I20260808 00:38:19.939298 20395 raft_consensus.cc:374] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:19.939324 20403 ts_tablet_manager.cc:1397] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:38:19.939340 20395 raft_consensus.cc:729] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a1cb3d536846bcb6aa4447f9e4ad03, State: Initialized, Role: FOLLOWER
I20260808 00:38:19.939457 20395 consensus_queue.cc:260] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.939738 20395 ts_tablet_manager.cc:1428] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:38:19.939767 20128 heartbeater.cc:503] Master 127.1.208.62:46715 was elected leader, sending a full tablet report...
I20260808 00:38:19.940356 20397 tablet_bootstrap.cc:492] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: No bootstrap required, opened a new log
I20260808 00:38:19.940431 20397 ts_tablet_manager.cc:1397] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 00:38:19.941155 20403 raft_consensus.cc:348] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.941345 20403 raft_consensus.cc:374] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:19.941391 20403 raft_consensus.cc:729] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2912822bec8a42439a3df305dff78f21, State: Initialized, Role: FOLLOWER
I20260808 00:38:19.941505 20403 consensus_queue.cc:260] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.941789 20369 heartbeater.cc:503] Master 127.1.208.62:46715 was elected leader, sending a full tablet report...
I20260808 00:38:19.941787 20403 ts_tablet_manager.cc:1428] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:38:19.942404 20397 raft_consensus.cc:348] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.942526 20397 raft_consensus.cc:374] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:38:19.942549 20397 raft_consensus.cc:729] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259c443bdee848659f5c49f58b2f3d3a, State: Initialized, Role: FOLLOWER
I20260808 00:38:19.942657 20397 consensus_queue.cc:260] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.942961 20397 ts_tablet_manager.cc:1428] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:38:19.942976 19901 heartbeater.cc:503] Master 127.1.208.62:46715 was elected leader, sending a full tablet report...
I20260808 00:38:19.956085 20410 raft_consensus.cc:482] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:38:19.956208 20410 raft_consensus.cc:504] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.956566 20410 leader_election.cc:288] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16a1cb3d536846bcb6aa4447f9e4ad03 (127.1.208.2:35207), 2912822bec8a42439a3df305dff78f21 (127.1.208.3:36967)
I20260808 00:38:19.960366 20394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.956816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60611 (local address 127.1.208.3:36967)
0808 00:38:19.956866 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.956868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.956877 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:38:19.956903 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.956905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.956916 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.956964 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:38:19.957136 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.957964 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.959419 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.960004 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.960070 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.960082 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.960249 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.960295 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:38:19.960384 20393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.956806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:40037 (local address 127.1.208.2:35207)
0808 00:38:19.956872 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.956873 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.956998 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:38:19.957032 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.957033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.957043 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.957082 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:38:19.957211 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.957956 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.959423 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.959995 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.960070 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.960082 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.960282 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.960325 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:38:19.960393 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.956716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36967 (local address 127.1.208.1:60611)
0808 00:38:19.956764 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.956776 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.956814 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.956990 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.956991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.956993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:38:19.957068 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.957070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.958034 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.958035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.959302 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.959305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.960074 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.960075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.960180 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.960182 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.960235 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.960292 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":41}
I20260808 00:38:19.960399 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.956716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35207 (local address 127.1.208.1:40037)
0808 00:38:19.956930 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:19.956938 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:38:19.956991 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:38:19.957097 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:38:19.957098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:38:19.957099 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:38:19.957156 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.957157 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.958034 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.958035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.959302 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:38:19.959305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.960074 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.960075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:38:19.960175 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.960180 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:38:19.960235 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:38:19.960292 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":122,"thread_start_us":115,"threads_started":1}
I20260808 00:38:19.960803 20050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24315bb8065846d6b736f51b84719391" candidate_uuid: "259c443bdee848659f5c49f58b2f3d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" is_pre_election: true
I20260808 00:38:19.960804 20297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24315bb8065846d6b736f51b84719391" candidate_uuid: "259c443bdee848659f5c49f58b2f3d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2912822bec8a42439a3df305dff78f21" is_pre_election: true
I20260808 00:38:19.960984 20297 raft_consensus.cc:2454] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259c443bdee848659f5c49f58b2f3d3a in term 0.
I20260808 00:38:19.960984 20050 raft_consensus.cc:2454] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259c443bdee848659f5c49f58b2f3d3a in term 0.
I20260808 00:38:19.961234 19717 leader_election.cc:302] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259c443bdee848659f5c49f58b2f3d3a, 2912822bec8a42439a3df305dff78f21; no voters:
I20260808 00:38:19.961390 20410 raft_consensus.cc:2781] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:38:19.961467 20410 raft_consensus.cc:482] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:38:19.961488 20410 raft_consensus.cc:3037] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:19.962342 20410 raft_consensus.cc:504] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.962484 20410 leader_election.cc:288] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [CANDIDATE]: Term 1 election: Requested vote from peers 16a1cb3d536846bcb6aa4447f9e4ad03 (127.1.208.2:35207), 2912822bec8a42439a3df305dff78f21 (127.1.208.3:36967)
I20260808 00:38:19.962628 20050 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24315bb8065846d6b736f51b84719391" candidate_uuid: "259c443bdee848659f5c49f58b2f3d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03"
I20260808 00:38:19.962731 20050 raft_consensus.cc:3037] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:19.962719 20297 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24315bb8065846d6b736f51b84719391" candidate_uuid: "259c443bdee848659f5c49f58b2f3d3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2912822bec8a42439a3df305dff78f21"
I20260808 00:38:19.962800 20297 raft_consensus.cc:3037] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:38:19.963445 20050 raft_consensus.cc:2454] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259c443bdee848659f5c49f58b2f3d3a in term 1.
I20260808 00:38:19.963596 20297 raft_consensus.cc:2454] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259c443bdee848659f5c49f58b2f3d3a in term 1.
I20260808 00:38:19.963619 19713 leader_election.cc:302] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16a1cb3d536846bcb6aa4447f9e4ad03, 259c443bdee848659f5c49f58b2f3d3a; no voters:
I20260808 00:38:19.963766 20410 raft_consensus.cc:2781] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:38:19.963923 20410 raft_consensus.cc:686] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [term 1 LEADER]: Becoming Leader. State: Replica: 259c443bdee848659f5c49f58b2f3d3a, State: Running, Role: LEADER
I20260808 00:38:19.964013 20410 consensus_queue.cc:237] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } }
I20260808 00:38:19.964854 19580 catalog_manager.cc:5495] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a reported cstate change: term changed from 0 to 1, leader changed from <none> to 259c443bdee848659f5c49f58b2f3d3a (127.1.208.1). New cstate: current_term: 1 leader_uuid: "259c443bdee848659f5c49f58b2f3d3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "259c443bdee848659f5c49f58b2f3d3a" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35177 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 } health_report { overall_health: UNKNOWN } } }
I20260808 00:38:19.976243 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:19.974139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37172 (local address 127.1.208.1:35177)
0808 00:38:19.974194 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:38:19.974196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:38:19.974203 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:38:19.974239 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:38:19.974240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:38:19.974260 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:38:19.974318 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:38:19.974440 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.974904 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.975594 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:38:19.975795 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:38:19.975848 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:38:19.975963 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:38:19.976059 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:38:19.976116 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:38:19.976129 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 00:38:19.976146 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:38:19.978130 20297 raft_consensus.cc:1261] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 1 FOLLOWER]: Refusing update from remote peer 259c443bdee848659f5c49f58b2f3d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:38:19.978130 20050 raft_consensus.cc:1261] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 1 FOLLOWER]: Refusing update from remote peer 259c443bdee848659f5c49f58b2f3d3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:38:19.978478 20410 consensus_queue.cc:1035] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2912822bec8a42439a3df305dff78f21" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:38:19.978631 20415 consensus_queue.cc:1035] T 24315bb8065846d6b736f51b84719391 P 259c443bdee848659f5c49f58b2f3d3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "16a1cb3d536846bcb6aa4447f9e4ad03" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:38:19.981002 20421 mvcc.cc:205] Tried to move back new op lower bound from 7316068351907524608 to 7316068351853142016. Current Snapshot: MvccSnapshot[applied={T|T < 7316068351907524608}]
I20260808 00:38:19.981209 20424 mvcc.cc:205] Tried to move back new op lower bound from 7316068351907524608 to 7316068351853142016. Current Snapshot: MvccSnapshot[applied={T|T < 7316068351907524608}]
I20260808 00:38:19.981501 20429 mvcc.cc:205] Tried to move back new op lower bound from 7316068351907524608 to 7316068351853142016. Current Snapshot: MvccSnapshot[applied={T|T < 7316068351907524608}]
I20260808 00:38:19.983050 19579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:38:19.983135 19579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:38:19.983876 19579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8b457430029496cadedcd70049a83c1: Sending DeleteTablet for 3 replicas of tablet 24315bb8065846d6b736f51b84719391
I20260808 00:38:19.984174 19791 tablet_service.cc:1514] Processing DeleteTablet for tablet 24315bb8065846d6b736f51b84719391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:38:19 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:37156
I20260808 00:38:19.984283 20257 tablet_service.cc:1514] Processing DeleteTablet for tablet 24315bb8065846d6b736f51b84719391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:38:19 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:56076
I20260808 00:38:19.984282 20017 tablet_service.cc:1514] Processing DeleteTablet for tablet 24315bb8065846d6b736f51b84719391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:38:19 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:42340
I20260808 00:38:19.984472 20442 tablet_replica.cc:330] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: stopping tablet replica
I20260808 00:38:19.984563 20442 raft_consensus.cc:2229] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:38:19.984665 20442 raft_consensus.cc:2258] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:38:19.984719 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 19685
I20260808 00:38:19.991832 20442 ts_tablet_manager.cc:1905] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:38:19.993468 20442 ts_tablet_manager.cc:1918] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:38:19.993541 20442 log.cc:1196] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/24315bb8065846d6b736f51b84719391
I20260808 00:38:19.993769 20442 ts_tablet_manager.cc:1939] T 24315bb8065846d6b736f51b84719391 P 2912822bec8a42439a3df305dff78f21: Deleting consensus metadata
I20260808 00:38:19.994189 19561 catalog_manager.cc:4841] TS 2912822bec8a42439a3df305dff78f21 (127.1.208.3:36967): tablet 24315bb8065846d6b736f51b84719391 (table test-table [id=b034eb8837b2476497a7000ccf412f45]) successfully deleted
I20260808 00:38:20.001128 20443 tablet_replica.cc:330] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: stopping tablet replica
I20260808 00:38:20.001319 20443 raft_consensus.cc:2229] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:38:20.001474 20443 raft_consensus.cc:2258] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:38:20.002146 20443 ts_tablet_manager.cc:1905] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:38:20.003659 20443 ts_tablet_manager.cc:1918] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260808 00:38:20.003657 19556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:35177 (error 108)
I20260808 00:38:20.003744 20443 log.cc:1196] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/24315bb8065846d6b736f51b84719391
I20260808 00:38:20.003870 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 19908
I20260808 00:38:20.004055 20443 ts_tablet_manager.cc:1939] T 24315bb8065846d6b736f51b84719391 P 16a1cb3d536846bcb6aa4447f9e4ad03: Deleting consensus metadata
I20260808 00:38:20.006700 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:20.004152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35177 (local address 127.0.0.1:37188)
0808 00:38:20.006597 (+ 2445us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:20.006652 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35177: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2414}
W20260808 00:38:20.007332 19556 catalog_manager.cc:4584] TS 259c443bdee848659f5c49f58b2f3d3a (127.1.208.1:35177): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24315bb8065846d6b736f51b84719391: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35177: connect: Connection refused (error 111)
I20260808 00:38:20.014729 20390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:38:20.014614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35207 (local address 127.0.0.1:42352)
0808 00:38:20.014668 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:38:20.014697 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.2:35207: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:38:20.014730 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 20139
I20260808 00:38:20.037521 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 19525
2026-08-08T00:38:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2067 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:38:20 dist-test-slave-gjw7 krb5kdc[20448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:38:20 dist-test-slave-gjw7 krb5kdc[20448](info): set up 2 sockets
Aug 08 00:38:20 dist-test-slave-gjw7 krb5kdc[20448](info): commencing operation
krb5kdc: starting...
W20260808 00:38:21.217041 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:38:21 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:38:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:38:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangeradmin_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangeradmin_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangerlookup_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangerlookup_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
I20260808 00:38:21.380194 1856 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknSGkLY/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/postgres -l logfile start
2026-08-08 00:38:23.746 UTC [20502] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:38:23.746 UTC [20502] LOG: listening on IPv4 address "127.1.208.20", port 49081
2026-08-08 00:38:23.750 UTC [20502] LOG: listening on Unix socket "/tmp/.s.PGSQL.49081"
2026-08-08 00:38:23.770 UTC [20506] LOG: database system was shut down at 2026-08-08 00:38:22 UTC
2026-08-08 00:38:23.774 UTC [20502] LOG: database system is ready to accept connections
I20260808 00:38:24.855178 1856 mini_postgres.cc:96] Postgres bound to 49081
2026-08-08 00:38:24.862 UTC [20518] FATAL: database "slave" does not exist
127.1.208.20:49081 - accepting connections
I20260808 00:38:24.863076 1856 mini_ranger.cc:161] Starting Ranger...
I20260808 00:38:24.877882 1856 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:38:25.324404 1856 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:38:25.324491 1856 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin
2026-08-08 00:38:25,610 [I] DB FLAVOR :POSTGRES
2026-08-08 00:38:25,611 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:38:25,611 [I] Checking connection..
2026-08-08 00:38:25,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:38:25,877 [I] Checking connection passed.
2026-08-08 00:38:25,877 [I] --------- Verifying version history table ---------
2026-08-08 00:38:25,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:38:26,124 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:38:26,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:38:26,388 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:38:26,389 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:38:26,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:38:26.628 UTC [20626] WARNING: there is no transaction in progress
2026-08-08 00:38:26,636 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:38:26,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:38:26,896 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:38:26,896 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:38:26,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:38:27,134 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:38:27,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-08 00:38:27,627 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:38:27,893 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:38:27,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:38:28,147 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:38:28,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:38:28,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:38:28,661 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:38:28,661 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:38:29.628 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.644 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.659 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.679 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.694 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.711 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.768 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.774 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.788 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.797 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.807 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.820 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.830 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.836 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.842 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:29.847 UTC [20807] WARNING: there is no transaction in progress
2026-08-08 00:38:30,443 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:38:30,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:38:30,671 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:38:30,672 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:38:30,917 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:38:30,917 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:38:31,156 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:38:31,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:38:31,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:38:31,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-08 00:38:31,918 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:38:31,919 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:38:32,183 [I] DB_PATCHES have already been applied
I20260808 00:38:32.191780 1856 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknSGkLY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:38:32.191854 1856 mini_ranger.cc:193] Using host: 127.1.208.20
I20260808 00:38:32.196108 1856 mini_ranger.cc:227] Ranger admin URL: http://127.1.208.20:39143
Aug 08, 2026 12:38:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:38:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:38:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:38:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:38:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:38:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:38:33 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:38:33.537256 1856 mini_ranger.cc:160] Time spent starting Ranger: real 8.674s user 0.004s sys 0.007s
Aug 08, 2026 12:38:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:38:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 12:38:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:38:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:38:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:38:54.679425 1856 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangerkms_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/rangerkms_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:38:54 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149534, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:38:54.764992 1856 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:38:54.778116 1856 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:38:55.506036 1856 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:38:55.506102 1856 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms
I20260808 00:38:55.506114 1856 mini_ranger_kms.cc:227] Using postgres at 127.1.208.20:49081
2026-08-08 00:38:56,691 [I] DB FLAVOR :POSTGRES
2026-08-08 00:38:56,691 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:38:56,691 [I] Checking connection
2026-08-08 00:38:56,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:38:57,083 [I] connection success
2026-08-08 00:38:57,083 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:38:57,083 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:38:57,083 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:38:57,334 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:38:57,335 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:38:57,335 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:38:57,335 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:49081/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:38:57,605 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:38:58.044263 1856 mini_ranger_kms.cc:310] Created kms service
I20260808 00:38:58.210408 1856 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:38:58.253036 1856 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:38:58 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149534, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:38:58.606434 1856 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:38:58.606549 1856 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:38:58.606570 1856 mini_ranger_kms.cc:240] Using host: 127.1.208.20
I20260808 00:38:58.610765 1856 mini_ranger_kms.cc:276] Ranger KMS PID: 21105
I20260808 00:38:58.610894 1856 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.208.20:45867
Aug 08, 2026 12:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:38:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:39:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:39:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260808 00:39:01.130120 1856 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.365s user 0.008s sys 0.966s
I20260808 00:39:01.130224 1856 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:39:01.130308 1856 mini_ranger_kms.cc:401] 127.1.208.20:45867/kms/v1/keys
Aug 08 00:39:01 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149541, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 00:39:01 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149541, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
Aug 08 00:39:02 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149541, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
Aug 08 00:39:04 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:39:04.784001 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34509
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:34083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34509
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:45867/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:04.964567 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:04.964864 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:04.964898 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:04.966678 21181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:04.966738 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:04.966756 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:04.966770 21181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:04.966789 21181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:04.969368 21181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.208.20:45867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34509
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:04.969632 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:04.969887 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:04.976668 21188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:04.976675 21190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:04.976711 21194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:04.976890 21181 server_base.cc:1034] running on GCE node
I20260808 00:39:04.977250 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:04.977561 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:04.978703 21181 hybrid_clock.cc:648] HybridClock initialized: now 1786149544978691 us; error 32 us; skew 500 ppm
Aug 08 00:39:04 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:04.981914 21181 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:39:04.982359 21181 webserver.cc:460] Webserver started at http://127.1.208.62:41343/ using document root <none> and password file <none>
I20260808 00:39:04.982551 21181 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:04.982589 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:04.982707 21181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:04 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149544, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:05.048035 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "822e1c5bfc764601b109d960aef03d7e"
format_stamp: "Formatted at 2026-08-08 00:39:05 on dist-test-slave-gjw7"
server_key: "27258dbf2112c55b2a17eceea0068765"
server_key_iv: "4133228be42daee02a22589fe0029d41"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:05.048599 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "822e1c5bfc764601b109d960aef03d7e"
format_stamp: "Formatted at 2026-08-08 00:39:05 on dist-test-slave-gjw7"
server_key: "27258dbf2112c55b2a17eceea0068765"
server_key_iv: "4133228be42daee02a22589fe0029d41"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:05.050843 21181 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 00:39:05.066172 21204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:05.066395 21181 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:39:05.066509 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "822e1c5bfc764601b109d960aef03d7e"
format_stamp: "Formatted at 2026-08-08 00:39:05 on dist-test-slave-gjw7"
server_key: "27258dbf2112c55b2a17eceea0068765"
server_key_iv: "4133228be42daee02a22589fe0029d41"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:05.066612 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:05.080067 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:05.082216 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:05.082437 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:05.096032 21181 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34509
I20260808 00:39:05.096170 21286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34509 every 8 connection(s)
I20260808 00:39:05.096618 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:39:05.097760 21288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:05.101446 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 21181
I20260808 00:39:05.101457 21288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: Bootstrap starting.
I20260808 00:39:05.101548 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:39:05.101751 1856 external_mini_cluster.cc:1428] Setting key 0d0fa7950b38ef71003dc6c48a2cad4f
I20260808 00:39:05.102878 21288 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:05.103652 21288 log.cc:824] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:05.104660 21288 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: No bootstrap required, opened a new log
Aug 08 00:39:05 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:05.109238 21288 raft_consensus.cc:348] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } }
I20260808 00:39:05.109440 21288 raft_consensus.cc:374] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:05.109510 21288 raft_consensus.cc:729] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822e1c5bfc764601b109d960aef03d7e, State: Initialized, Role: FOLLOWER
I20260808 00:39:05.109653 21288 consensus_queue.cc:260] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } }
I20260808 00:39:05.109727 21288 raft_consensus.cc:388] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:39:05.109754 21288 raft_consensus.cc:482] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:39:05.109815 21288 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:05.110679 21288 raft_consensus.cc:504] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } }
I20260808 00:39:05.110795 21288 leader_election.cc:302] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 822e1c5bfc764601b109d960aef03d7e; no voters:
I20260808 00:39:05.111008 21288 leader_election.cc:288] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:39:05.111248 21297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:39:05.111295 21288 sys_catalog.cc:564] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:05.111600 21297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [term 1 LEADER]: Becoming Leader. State: Replica: 822e1c5bfc764601b109d960aef03d7e, State: Running, Role: LEADER
I20260808 00:39:05.111689 21297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } }
I20260808 00:39:05.112684 21299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 822e1c5bfc764601b109d960aef03d7e. Latest consensus state: current_term: 1 leader_uuid: "822e1c5bfc764601b109d960aef03d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } } }
I20260808 00:39:05.112843 21299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:05.113142 21298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "822e1c5bfc764601b109d960aef03d7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "822e1c5bfc764601b109d960aef03d7e" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34509 } } }
I20260808 00:39:05.113327 21298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:05.114550 21291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:05.102337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46280 (local address 127.1.208.62:34509)
0808 00:39:05.102733 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:05.102744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:05.102804 (+ 60us) server_negotiation.cc:407] Connection header received
0808 00:39:05.103782 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:05.103792 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:05.103862 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:05.104013 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:05.104853 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:05.106001 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:05.106791 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:05.107356 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:05.107461 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:05.109946 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:05.109949 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:05.109950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:05.109960 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:05.112138 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:05.112603 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:05.112606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:05.112606 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:05.112811 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:05.113081 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:05.113082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:05.113082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:05.113422 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:05.114079 (+ 657us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:05.114155 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:39:05.114281 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":221,"threads_started":1}
I20260808 00:39:05.114954 21288 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:39:05.120007 21319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:39:05.122354 21319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:39:05.125002 21319 catalog_manager.cc:1347] Generated new cluster ID: fa10ced5decb46e4b372808a4c611516
I20260808 00:39:05.125048 21319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:39:05.145370 21319 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:39:05.146263 21319 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:39:05.153672 21319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: Generated new TSK 0
I20260808 00:39:05.153908 21319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:39:06.184448 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:45867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:34083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:06.317524 21366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:06.317742 21366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:06.317777 21366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:06.319022 21366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:06.319068 21366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:06.319104 21366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:39:06.320823 21366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.208.20:45867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:06.321056 21366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:06.321359 21366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:06.326891 21379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:06.326980 21382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:06.326948 21378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:06.327181 21366 server_base.cc:1034] running on GCE node
I20260808 00:39:06.327425 21366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:06.327643 21366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:06.328787 21366 hybrid_clock.cc:648] HybridClock initialized: now 1786149546328774 us; error 32 us; skew 500 ppm
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:06.331511 21366 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:39:06.331954 21366 webserver.cc:460] Webserver started at http://127.1.208.1:44093/ using document root <none> and password file <none>
I20260808 00:39:06.332159 21366 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:06.332206 21366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:06.332338 21366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:06.347743 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "9822561d57824a38bd5ba52879681760"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "1fb06538acfa7ec101e7fc12b79f758b"
server_key_iv: "bfb5d7cd11292252743eb904c094beb6"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.348630 21366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "9822561d57824a38bd5ba52879681760"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "1fb06538acfa7ec101e7fc12b79f758b"
server_key_iv: "bfb5d7cd11292252743eb904c094beb6"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.351797 21366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 00:39:06.366895 21394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.367134 21366 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 00:39:06.367240 21366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "9822561d57824a38bd5ba52879681760"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "1fb06538acfa7ec101e7fc12b79f758b"
server_key_iv: "bfb5d7cd11292252743eb904c094beb6"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.367337 21366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:06.420060 21366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:06.422688 21366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:06.422878 21366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:06.423255 21366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:06.423753 21366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:06.423808 21366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.423914 21366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:06.423950 21366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.444608 21366 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:39513
I20260808 00:39:06.445268 21366 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:39:06.444643 21573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:39513 every 8 connection(s)
I20260808 00:39:06.454844 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 21366
I20260808 00:39:06.454953 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:39:06.455164 1856 external_mini_cluster.cc:1428] Setting key 359a4f1286d054eb2bcdd6389db55fa1
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:06.462491 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:06.446257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34509 (local address 127.1.208.1:34665)
0808 00:39:06.447085 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:06.447141 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:06.448278 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:06.450659 (+ 2381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:06.450665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:06.451168 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:06.451634 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:06.451642 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.452821 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.452822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:06.454324 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:06.454328 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.454970 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.454972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:06.455189 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:06.456266 (+ 1077us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:06.456277 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:06.458382 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:06.460599 (+ 2217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:06.460606 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:06.460608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:06.460914 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:06.461157 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:06.461158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:06.461158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:06.461336 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:06.461641 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:06.461647 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:06.461909 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 00:39:06.462012 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 00:39:06.462172 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"thread_start_us":233,"threads_started":1}
I20260808 00:39:06.463080 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:06.446424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34665 (local address 127.1.208.62:34509)
0808 00:39:06.448503 (+ 2079us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:06.448506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:06.448514 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:06.448543 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:06.448544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:06.448556 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:06.448628 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:06.451772 (+ 3144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.452693 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.454428 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.454821 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.454890 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:06.458488 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:06.458490 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:06.458490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:06.458498 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:06.460505 (+ 2007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:06.461010 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:06.461011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:06.461012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:06.461087 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:06.461413 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:06.461414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:06.461415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:06.461556 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:06.461614 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:06.462883 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0808 00:39:06.462973 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2054,"thread_start_us":69,"threads_started":1}
I20260808 00:39:06.463876 21577 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34509
I20260808 00:39:06.464035 21577 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:06.464286 21577 heartbeater.cc:511] Master 127.1.208.62:34509 requested a full tablet report, sending...
I20260808 00:39:06.464900 21230 ts_manager.cc:194] Registered new tserver with Master: 9822561d57824a38bd5ba52879681760 (127.1.208.1:39513)
I20260808 00:39:06.465811 21230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:34665
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:39:06.495688 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:45867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:34083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:06.665951 21586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:06.666323 21586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:06.666433 21586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:06.668304 21586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:06.668421 21586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:06.668514 21586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:39:06.671131 21586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.208.20:45867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:06.671674 21586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:06.672072 21586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:06.681708 21596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:06.682109 21594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:06.683295 21599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:06.684266 21586 server_base.cc:1034] running on GCE node
I20260808 00:39:06.684692 21586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:06.684988 21586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:06.692373 21586 hybrid_clock.cc:648] HybridClock initialized: now 1786149546692339 us; error 46 us; skew 500 ppm
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:06.697186 21586 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:39:06.697844 21586 webserver.cc:460] Webserver started at http://127.1.208.2:42293/ using document root <none> and password file <none>
I20260808 00:39:06.698172 21586 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:06.698312 21586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:06.698503 21586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:06.722905 21586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "3eaee61750900b4478150d9a415fff9c"
server_key_iv: "3aca219f786ae634e8af24b5f131bc30"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.723723 21586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "3eaee61750900b4478150d9a415fff9c"
server_key_iv: "3aca219f786ae634e8af24b5f131bc30"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.726063 21586 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:39:06.742532 21609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.742766 21586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:39:06.742859 21586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c"
format_stamp: "Formatted at 2026-08-08 00:39:06 on dist-test-slave-gjw7"
server_key: "3eaee61750900b4478150d9a415fff9c"
server_key_iv: "3aca219f786ae634e8af24b5f131bc30"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:06.742959 21586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:06.773072 21586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:06.775918 21586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:06.776275 21586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:06.776715 21586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:06.777179 21586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:06.777275 21586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.777379 21586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:06.777443 21586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:06.794410 21586 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:33771
I20260808 00:39:06.795179 21586 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:39:06.795845 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 21586
I20260808 00:39:06.795979 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:39:06.796200 1856 external_mini_cluster.cc:1428] Setting key 1484cc3d7aba216e523f27b06b75d5b6
I20260808 00:39:06.798445 21801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:33771 every 8 connection(s)
Aug 08 00:39:06 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149546, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
I20260808 00:39:06.813755 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:06.799490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:33957 (local address 127.1.208.62:34509)
0808 00:39:06.799542 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:06.799545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:06.799631 (+ 86us) server_negotiation.cc:407] Connection header received
0808 00:39:06.800902 (+ 1271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:06.800904 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:06.800919 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:06.800954 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:06.802248 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.803024 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.804830 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.805241 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.805326 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:06.809701 (+ 4375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:06.809703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:06.809703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:06.809712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:06.811624 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:06.812239 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:06.812241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:06.812241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:06.812314 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:06.812649 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:06.812650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:06.812651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:06.812810 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:06.812864 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:06.813509 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 00:39:06.813613 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:39:06.814762 21807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:06.799146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34509 (local address 127.1.208.2:33957)
0808 00:39:06.799450 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:06.799508 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:06.800760 (+ 1252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:06.801195 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:06.801200 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:06.801684 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:06.802106 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:06.802115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.803159 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.803160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:06.804715 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:06.804720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:06.805463 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:06.805464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:06.805813 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:06.806875 (+ 1062us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:06.806886 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:06.809567 (+ 2681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:06.811735 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:06.811743 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:06.811746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:06.812112 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:06.812398 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:06.812398 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:06.812399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:06.812549 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:06.812960 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:06.812968 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:06.813247 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 00:39:06.814193 (+ 946us) client_negotiation.cc:240] Negotiation successful
0808 00:39:06.814386 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":165,"threads_started":1}
I20260808 00:39:06.815587 21802 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34509
I20260808 00:39:06.815702 21802 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:06.815915 21802 heartbeater.cc:511] Master 127.1.208.62:34509 requested a full tablet report, sending...
I20260808 00:39:06.816416 21230 ts_manager.cc:194] Registered new tserver with Master: fd5b526b6f7245ca8b1430a0b94d5a6c (127.1.208.2:33771)
I20260808 00:39:06.816916 21230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:33957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:39:06.843971 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:45867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:34083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:07.003469 21814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:07.003866 21814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:07.003983 21814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:07.005913 21814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:07.006129 21814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:07.006243 21814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:39:07.008685 21814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.208.20:45867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:07.009188 21814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:07.009567 21814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:07.021101 21828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:07.022691 21824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:07.022890 21825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:07.024029 21814 server_base.cc:1034] running on GCE node
I20260808 00:39:07.024380 21814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:07.024662 21814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:07.025830 21814 hybrid_clock.cc:648] HybridClock initialized: now 1786149547025804 us; error 36 us; skew 500 ppm
Aug 08 00:39:07 dist-test-slave-gjw7 krb5kdc[20448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:07.030589 21814 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:39:07.031298 21814 webserver.cc:460] Webserver started at http://127.1.208.3:37857/ using document root <none> and password file <none>
I20260808 00:39:07.031646 21814 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:07.031796 21814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:07.032006 21814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:07 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:07.046339 21814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "363177cf69034518855f0a8be1b7b876"
format_stamp: "Formatted at 2026-08-08 00:39:07 on dist-test-slave-gjw7"
server_key: "20c0b0486e7c1d22f25996b199f6c74d"
server_key_iv: "e2c7697a1c8e55b70eac22865d24102a"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:07.046855 21814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "363177cf69034518855f0a8be1b7b876"
format_stamp: "Formatted at 2026-08-08 00:39:07 on dist-test-slave-gjw7"
server_key: "20c0b0486e7c1d22f25996b199f6c74d"
server_key_iv: "e2c7697a1c8e55b70eac22865d24102a"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:07.048728 21814 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260808 00:39:07.061262 21838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:07.061480 21814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:07.061583 21814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "363177cf69034518855f0a8be1b7b876"
format_stamp: "Formatted at 2026-08-08 00:39:07 on dist-test-slave-gjw7"
server_key: "20c0b0486e7c1d22f25996b199f6c74d"
server_key_iv: "e2c7697a1c8e55b70eac22865d24102a"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:07.061676 21814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:07.072862 21814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:07.074879 21814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:07.075016 21814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:07.075415 21814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:07.075942 21814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:07.075989 21814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:07.076018 21814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:07.076040 21814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:07.096700 21814 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:34203
I20260808 00:39:07.097260 21814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:39:07.097458 22019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:34203 every 8 connection(s)
I20260808 00:39:07.097782 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 21814
I20260808 00:39:07.097885 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:39:07.098105 1856 external_mini_cluster.cc:1428] Setting key 0aea9a6244563708d873bc9bb3dced67
Aug 08 00:39:07 dist-test-slave-gjw7 krb5kdc[20448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149547, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:07.112258 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.100217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:46779 (local address 127.1.208.62:34509)
0808 00:39:07.100263 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.100265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.100337 (+ 72us) server_negotiation.cc:407] Connection header received
0808 00:39:07.101436 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.101438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.101451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.101527 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:07.102840 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.103681 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.105331 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.105617 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.105692 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.108779 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:07.108780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:07.108781 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:07.108789 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:07.110463 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:07.111019 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:07.111021 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:07.111021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:07.111093 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:07.111410 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:07.111411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:07.111412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:07.111557 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:07.111626 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.112062 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.112152 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:39:07.112510 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.099784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34509 (local address 127.1.208.3:46779)
0808 00:39:07.100171 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.100230 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.101264 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.101612 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.101617 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.102160 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:07.102735 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.102743 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.103787 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.103788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.105201 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.105204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.105706 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.105707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.105909 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.106681 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:07.106692 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:07.108677 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:07.110555 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:07.110562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:07.110565 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:07.110913 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:07.111177 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:07.111178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:07.111178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:07.111333 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:07.111636 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:07.111643 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:07.111899 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.112034 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.112209 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":218,"threads_started":1}
I20260808 00:39:07.113193 22020 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34509
I20260808 00:39:07.113336 22020 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:07.113561 22020 heartbeater.cc:511] Master 127.1.208.62:34509 requested a full tablet report, sending...
I20260808 00:39:07.114005 21230 ts_manager.cc:194] Registered new tserver with Master: 363177cf69034518855f0a8be1b7b876 (127.1.208.3:34203)
I20260808 00:39:07.114483 21230 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:46779
I20260808 00:39:07.121665 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:39:07.133823 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.125645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.1.208.62:34509)
0808 00:39:07.125694 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.125696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.125946 (+ 250us) server_negotiation.cc:407] Connection header received
0808 00:39:07.125980 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.125982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.125993 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.126059 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:07.126828 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.127559 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.128642 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.128930 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.128978 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.130744 (+ 1766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:07.130746 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:07.130746 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:07.130753 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:07.132363 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:07.132766 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:07.132767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:07.132767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:07.132829 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:07.133042 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:07.133043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:07.133043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:07.133180 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:07.133336 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.133583 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.133668 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:39:07.135732 21230 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:39:07.138415 21230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:39:07.158628 22042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.144600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39513 (local address 127.0.0.1:33348)
0808 00:39:07.147082 (+ 2482us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.147127 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.147192 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.149188 (+ 1996us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.149189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.149192 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:07.149270 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.149272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.150986 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.150987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.155208 (+ 4221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.155211 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.158303 (+ 3092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.158304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.158422 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.158440 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.158492 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.158542 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2424,"thread_start_us":177,"threads_started":1}
I20260808 00:39:07.160706 22041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.145931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:33771 (local address 127.0.0.1:45926)
0808 00:39:07.148756 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.148768 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.148828 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.152141 (+ 3313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.152143 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.152144 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:07.152180 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.152182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.155611 (+ 3429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.155612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.156795 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.156798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.160417 (+ 3619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.160418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.160481 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.160486 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.160533 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.160572 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2757,"thread_start_us":147,"threads_started":1}
I20260808 00:39:07.162015 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.145377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.1.208.1:39513)
0808 00:39:07.147348 (+ 1971us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.147354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.147363 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:39:07.147397 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.147399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.147415 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.147477 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:07.149377 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.150890 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.156894 (+ 6004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.157939 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.161740 (+ 3801us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.161829 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.161860 (+ 31us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.161929 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1940,"thread_start_us":101,"threads_started":1}
I20260808 00:39:07.162266 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.144941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:34203 (local address 127.0.0.1:37624)
0808 00:39:07.147864 (+ 2923us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.147873 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.147928 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.148386 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.148390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.148393 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:07.148542 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.148545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.150305 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.150306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.151693 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.151696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.161913 (+ 10217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.161914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.162039 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.162044 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.162105 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.162152 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2883,"thread_start_us":157,"threads_started":1}
I20260808 00:39:07.162662 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.145609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37624 (local address 127.1.208.3:34203)
0808 00:39:07.147764 (+ 2155us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.147769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.147935 (+ 166us) server_negotiation.cc:407] Connection header received
0808 00:39:07.148018 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.148020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.148039 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.148203 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:07.148641 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.150205 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.152005 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.161575 (+ 9570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.161659 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.161865 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.162486 (+ 621us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.162533 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2010,"thread_start_us":150,"threads_started":1}
I20260808 00:39:07.163270 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.146272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.1.208.2:33771)
0808 00:39:07.148112 (+ 1840us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.148116 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.148883 (+ 767us) server_negotiation.cc:407] Connection header received
0808 00:39:07.148963 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.148965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.148981 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.149050 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:07.152253 (+ 3203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.155539 (+ 3286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.159268 (+ 3729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.160195 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.160267 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.160380 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.163073 (+ 2693us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.163120 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1780,"thread_start_us":226,"threads_started":1}
I20260808 00:39:07.163533 21445 tablet_service.cc:1467] Processing CreateTablet for tablet db68da2d784f4976b32840e65b7d0f51 (DEFAULT_TABLE table=test-table [id=c717e0a529cd4f1fa519e2288008b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:07.163990 21445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db68da2d784f4976b32840e65b7d0f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:07.164188 21684 tablet_service.cc:1467] Processing CreateTablet for tablet db68da2d784f4976b32840e65b7d0f51 (DEFAULT_TABLE table=test-table [id=c717e0a529cd4f1fa519e2288008b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:07.164528 21684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db68da2d784f4976b32840e65b7d0f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:07.170037 21915 tablet_service.cc:1467] Processing CreateTablet for tablet db68da2d784f4976b32840e65b7d0f51 (DEFAULT_TABLE table=test-table [id=c717e0a529cd4f1fa519e2288008b9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:07.170408 21915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet db68da2d784f4976b32840e65b7d0f51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:07.176887 22049 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Bootstrap starting.
I20260808 00:39:07.178794 22049 tablet_bootstrap.cc:654] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:07.179733 22049 log.cc:824] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:07.180027 22052 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Bootstrap starting.
I20260808 00:39:07.181600 22053 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Bootstrap starting.
I20260808 00:39:07.181974 22052 tablet_bootstrap.cc:654] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:07.183414 22052 log.cc:824] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:07.183506 22053 tablet_bootstrap.cc:654] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:07.184661 22049 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: No bootstrap required, opened a new log
I20260808 00:39:07.184791 22049 ts_tablet_manager.cc:1397] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 00:39:07.185979 22053 log.cc:824] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:07.189936 22049 raft_consensus.cc:348] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.190248 22049 raft_consensus.cc:374] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:07.190414 22049 raft_consensus.cc:729] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9822561d57824a38bd5ba52879681760, State: Initialized, Role: FOLLOWER
I20260808 00:39:07.190663 22049 consensus_queue.cc:260] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.191274 22049 ts_tablet_manager.cc:1428] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 00:39:07.191474 21577 heartbeater.cc:503] Master 127.1.208.62:34509 was elected leader, sending a full tablet report...
I20260808 00:39:07.193423 22052 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: No bootstrap required, opened a new log
I20260808 00:39:07.193523 22052 ts_tablet_manager.cc:1397] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.000s
I20260808 00:39:07.196468 22052 raft_consensus.cc:348] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.196661 22052 raft_consensus.cc:374] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:07.196710 22052 raft_consensus.cc:729] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd5b526b6f7245ca8b1430a0b94d5a6c, State: Initialized, Role: FOLLOWER
I20260808 00:39:07.196856 22052 consensus_queue.cc:260] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
W20260808 00:39:07.197377 21580 tablet.cc:2369] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:39:07.197829 22052 ts_tablet_manager.cc:1428] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260808 00:39:07.197911 22053 tablet_bootstrap.cc:492] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: No bootstrap required, opened a new log
I20260808 00:39:07.198029 21802 heartbeater.cc:503] Master 127.1.208.62:34509 was elected leader, sending a full tablet report...
I20260808 00:39:07.198352 22053 ts_tablet_manager.cc:1397] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.003s
I20260808 00:39:07.202775 22053 raft_consensus.cc:348] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.202951 22053 raft_consensus.cc:374] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:07.202988 22053 raft_consensus.cc:729] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 363177cf69034518855f0a8be1b7b876, State: Initialized, Role: FOLLOWER
I20260808 00:39:07.203256 22053 consensus_queue.cc:260] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.203665 22053 ts_tablet_manager.cc:1428] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 00:39:07.203826 22020 heartbeater.cc:503] Master 127.1.208.62:34509 was elected leader, sending a full tablet report...
I20260808 00:39:07.205199 22065 raft_consensus.cc:482] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:39:07.205408 22065 raft_consensus.cc:504] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.210170 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.206020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39513 (local address 127.1.208.3:59485)
0808 00:39:07.206120 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.206131 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.206172 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.206507 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.206508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.206510 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:07.206676 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.206679 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.207638 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.207639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.208824 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.208828 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.209817 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.209818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.209974 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.209979 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.210035 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.210077 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260808 00:39:07.210538 22043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.206323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:59485 (local address 127.1.208.1:39513)
0808 00:39:07.206369 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.206371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.206378 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:39:07.206401 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.206401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.206410 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.206456 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:07.206764 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.207542 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.208921 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.209707 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.209773 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.209784 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.210452 (+ 668us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.210493 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:39:07.210834 21500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db68da2d784f4976b32840e65b7d0f51" candidate_uuid: "363177cf69034518855f0a8be1b7b876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9822561d57824a38bd5ba52879681760" is_pre_election: true
I20260808 00:39:07.210959 21500 raft_consensus.cc:2454] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 363177cf69034518855f0a8be1b7b876 in term 0.
I20260808 00:39:07.211241 21840 leader_election.cc:302] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 363177cf69034518855f0a8be1b7b876, 9822561d57824a38bd5ba52879681760; no voters:
I20260808 00:39:07.211668 22065 leader_election.cc:288] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9822561d57824a38bd5ba52879681760 (127.1.208.1:39513), fd5b526b6f7245ca8b1430a0b94d5a6c (127.1.208.2:33771)
I20260808 00:39:07.211800 22065 raft_consensus.cc:2781] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:39:07.211898 22065 raft_consensus.cc:482] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:39:07.211977 22065 raft_consensus.cc:3037] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:07.213565 22065 raft_consensus.cc:504] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.213860 21500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db68da2d784f4976b32840e65b7d0f51" candidate_uuid: "363177cf69034518855f0a8be1b7b876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9822561d57824a38bd5ba52879681760"
I20260808 00:39:07.213958 21500 raft_consensus.cc:3037] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:07.214936 22024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.211360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:33771 (local address 127.1.208.3:58927)
0808 00:39:07.211558 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.211570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.211611 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.211772 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:07.211773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:07.211775 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:07.211806 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.211807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.212806 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.212807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.213877 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:07.213880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.214726 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.214727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:07.214787 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.214791 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:39:07.214837 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:39:07.214869 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179}
I20260808 00:39:07.215137 22048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.211569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:58927 (local address 127.1.208.2:33771)
0808 00:39:07.211610 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.211611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.211619 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:07.211663 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.211664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.211674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.211729 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:07.211875 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.212582 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.213978 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.214596 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.214670 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.214682 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.215025 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.215070 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:39:07.215416 21500 raft_consensus.cc:2454] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 363177cf69034518855f0a8be1b7b876 in term 1.
I20260808 00:39:07.215600 21840 leader_election.cc:302] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 363177cf69034518855f0a8be1b7b876, 9822561d57824a38bd5ba52879681760; no voters:
I20260808 00:39:07.216104 21703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db68da2d784f4976b32840e65b7d0f51" candidate_uuid: "363177cf69034518855f0a8be1b7b876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c"
I20260808 00:39:07.216248 21703 raft_consensus.cc:3037] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:07.217458 21703 raft_consensus.cc:2454] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 363177cf69034518855f0a8be1b7b876 in term 1.
I20260808 00:39:07.216138 21700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "db68da2d784f4976b32840e65b7d0f51" candidate_uuid: "363177cf69034518855f0a8be1b7b876" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" is_pre_election: true
I20260808 00:39:07.217760 21700 raft_consensus.cc:2362] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 363177cf69034518855f0a8be1b7b876 in term 1. Re-sending same reply.
I20260808 00:39:07.218050 22065 leader_election.cc:288] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [CANDIDATE]: Term 1 election: Requested vote from peers 9822561d57824a38bd5ba52879681760 (127.1.208.1:39513), fd5b526b6f7245ca8b1430a0b94d5a6c (127.1.208.2:33771)
I20260808 00:39:07.218118 22065 raft_consensus.cc:2781] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:39:07.218180 22065 raft_consensus.cc:686] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 1 LEADER]: Becoming Leader. State: Replica: 363177cf69034518855f0a8be1b7b876, State: Running, Role: LEADER
I20260808 00:39:07.218268 22065 consensus_queue.cc:237] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } }
I20260808 00:39:07.219063 21229 catalog_manager.cc:5495] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 reported cstate change: term changed from 0 to 1, leader changed from <none> to 363177cf69034518855f0a8be1b7b876 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "363177cf69034518855f0a8be1b7b876" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "363177cf69034518855f0a8be1b7b876" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 34203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 } health_report { overall_health: UNKNOWN } } }
I20260808 00:39:07.242098 22044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.239580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.1.208.3:34203)
0808 00:39:07.239626 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:07.239628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:07.239636 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:07.239679 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:07.239680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:07.239701 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:07.239763 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:39:07.239939 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.240404 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.241143 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:07.241396 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:07.241457 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:07.241609 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:39:07.241726 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:39:07.241803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:07.241896 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:39:07.241917 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:39:07.244442 21700 raft_consensus.cc:1261] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 1 FOLLOWER]: Refusing update from remote peer 363177cf69034518855f0a8be1b7b876: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:07.244752 21500 raft_consensus.cc:1261] T db68da2d784f4976b32840e65b7d0f51 P 9822561d57824a38bd5ba52879681760 [term 1 FOLLOWER]: Refusing update from remote peer 363177cf69034518855f0a8be1b7b876: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:07.245213 22065 consensus_queue.cc:1035] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd5b526b6f7245ca8b1430a0b94d5a6c" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 33771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:39:07.245396 22072 consensus_queue.cc:1035] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9822561d57824a38bd5ba52879681760" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:39:07.249377 22071 mvcc.cc:205] Tried to move back new op lower bound from 7316068545510531072 to 7316068545406701568. Current Snapshot: MvccSnapshot[applied={T|T < 7316068545510531072}]
I20260808 00:39:07.249706 22075 mvcc.cc:205] Tried to move back new op lower bound from 7316068545510531072 to 7316068545406701568. Current Snapshot: MvccSnapshot[applied={T|T < 7316068545510531072}]
I20260808 00:39:07.251384 22073 mvcc.cc:205] Tried to move back new op lower bound from 7316068545510531072 to 7316068545406701568. Current Snapshot: MvccSnapshot[applied={T|T < 7316068545510531072}]
I20260808 00:39:07.253979 21229 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:39:07.254076 21229 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:39:07.254940 21229 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 822e1c5bfc764601b109d960aef03d7e: Sending DeleteTablet for 3 replicas of tablet db68da2d784f4976b32840e65b7d0f51
I20260808 00:39:07.255376 21445 tablet_service.cc:1514] Processing DeleteTablet for tablet db68da2d784f4976b32840e65b7d0f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:07 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33348
I20260808 00:39:07.256000 21684 tablet_service.cc:1514] Processing DeleteTablet for tablet db68da2d784f4976b32840e65b7d0f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:07 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45926
I20260808 00:39:07.256444 21915 tablet_service.cc:1514] Processing DeleteTablet for tablet db68da2d784f4976b32840e65b7d0f51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:07 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:37624
I20260808 00:39:07.257025 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 21366
I20260808 00:39:07.258004 22085 tablet_replica.cc:330] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: stopping tablet replica
I20260808 00:39:07.258179 22085 raft_consensus.cc:2229] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:39:07.260329 22085 raft_consensus.cc:2258] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:39:07.260859 22085 ts_tablet_manager.cc:1905] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:39:07.263307 22085 ts_tablet_manager.cc:1918] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:39:07.264636 22085 log.cc:1196] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/db68da2d784f4976b32840e65b7d0f51
I20260808 00:39:07.264987 22085 ts_tablet_manager.cc:1939] T db68da2d784f4976b32840e65b7d0f51 P fd5b526b6f7245ca8b1430a0b94d5a6c: Deleting consensus metadata
I20260808 00:39:07.265486 21206 catalog_manager.cc:4841] TS fd5b526b6f7245ca8b1430a0b94d5a6c (127.1.208.2:33771): tablet db68da2d784f4976b32840e65b7d0f51 (table test-table [id=c717e0a529cd4f1fa519e2288008b9b1]) successfully deleted
W20260808 00:39:07.268916 21205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:39513 (error 108)
I20260808 00:39:07.269127 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 21586
I20260808 00:39:07.272372 22047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:07.272220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:39513 (local address 127.0.0.1:33364)
0808 00:39:07.272273 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.272334 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:39513: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260808 00:39:07.272503 21205 catalog_manager.cc:4584] TS 9822561d57824a38bd5ba52879681760 (127.1.208.1:39513): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet db68da2d784f4976b32840e65b7d0f51: Network error: Client connection negotiation failed: client connection to 127.1.208.1:39513: connect: Connection refused (error 111)
I20260808 00:39:07.286269 22086 tablet_replica.cc:330] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: stopping tablet replica
I20260808 00:39:07.286497 22086 raft_consensus.cc:2229] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:39:07.286734 22086 raft_consensus.cc:2258] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:39:07.287375 22086 ts_tablet_manager.cc:1905] T db68da2d784f4976b32840e65b7d0f51 P 363177cf69034518855f0a8be1b7b876: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:39:07.289567 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 21814
W20260808 00:39:07.304320 22047 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:39:07.304039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:34203 (local address 127.0.0.1:37648)
0808 00:39:07.304090 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:07.304101 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:07.304139 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:07.304255 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.3:34203: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:39:07.304919 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 21181
I20260808 00:39:07.316144 1856 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:39:07.659549 1856 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:39:07.659667 1856 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:39:08.003093 1856 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:39:08.003 UTC [20502] LOG: received smart shutdown request
2026-08-08 00:39:08.006 UTC [20502] LOG: background worker "logical replication launcher" (PID 20515) exited with exit code 1
2026-08-08 00:39:08.007 UTC [20509] LOG: shutting down
2026-08-08 00:39:08.143 UTC [20502] LOG: database system is shut down
2026-08-08T00:39:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48185 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:39:08 dist-test-slave-gjw7 krb5kdc[22102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:39:08 dist-test-slave-gjw7 krb5kdc[22102](info): set up 2 sockets
Aug 08 00:39:08 dist-test-slave-gjw7 krb5kdc[22102](info): commencing operation
krb5kdc: starting...
W20260808 00:39:09.393060 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:39:09 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:39:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:39:09Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangeradmin_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangeradmin_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangerlookup_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangerlookup_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
I20260808 00:39:09.547669 1856 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasknSGkLY/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/postgres -l logfile start
2026-08-08 00:39:11.786 UTC [22149] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 00:39:11.786 UTC [22149] LOG: listening on IPv4 address "127.1.208.20", port 38509
2026-08-08 00:39:11.789 UTC [22149] LOG: listening on Unix socket "/tmp/.s.PGSQL.38509"
2026-08-08 00:39:11.808 UTC [22154] LOG: database system was shut down at 2026-08-08 00:39:10 UTC
2026-08-08 00:39:11.812 UTC [22149] LOG: database system is ready to accept connections
I20260808 00:39:12.898101 1856 mini_postgres.cc:96] Postgres bound to 38509
2026-08-08 00:39:12.905 UTC [22170] FATAL: database "slave" does not exist
127.1.208.20:38509 - accepting connections
I20260808 00:39:12.905531 1856 mini_ranger.cc:161] Starting Ranger...
I20260808 00:39:12.920140 1856 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 00:39:13.314008 1856 mini_ranger.cc:87] Created ranger Postgres database
I20260808 00:39:13.314085 1856 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin
2026-08-08 00:39:13,591 [I] DB FLAVOR :POSTGRES
2026-08-08 00:39:13,592 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:39:13,592 [I] Checking connection..
2026-08-08 00:39:13,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 00:39:13,839 [I] Checking connection passed.
2026-08-08 00:39:13,839 [I] --------- Verifying version history table ---------
2026-08-08 00:39:13,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:39:14,094 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:39:14,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:39:14,356 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 00:39:14,356 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 00:39:14,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 00:39:14.609 UTC [22285] WARNING: there is no transaction in progress
2026-08-08 00:39:14,616 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 00:39:14,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 00:39:14,877 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 00:39:14,877 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:39:14,877 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 00:39:15,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 00:39:15,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-gjw7.c.gcp-upstream.internal','N') ;"
2026-08-08 00:39:15,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:39:15,888 [I] Table x_portal_user does not exist in database ranger
2026-08-08 00:39:15,889 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:39:16,148 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 00:39:16,148 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:39:16,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:39:16,657 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 00:39:16,657 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 00:39:17.548 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.563 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.577 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.592 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.605 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.621 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.672 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.677 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.690 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.699 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.708 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.721 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.729 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.735 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.741 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:17.747 UTC [22462] WARNING: there is no transaction in progress
2026-08-08 00:39:18,307 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 00:39:18,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 00:39:18,561 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 00:39:18,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 00:39:18,801 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 00:39:18,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 00:39:19,081 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 00:39:19,082 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 00:39:19,346 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 00:39:19,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-gjw7.c.gcp-upstream.internal';"
2026-08-08 00:39:19,834 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 00:39:19,834 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 00:39:20,089 [I] DB_PATCHES have already been applied
I20260808 00:39:20.095139 1856 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasknSGkLY/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 00:39:20.095198 1856 mini_ranger.cc:193] Using host: 127.1.208.20
I20260808 00:39:20.099028 1856 mini_ranger.cc:227] Ranger admin URL: http://127.1.208.20:57229
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 12:39:20 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 00:39:21.361042 1856 mini_ranger.cc:160] Time spent starting Ranger: real 8.456s user 0.004s sys 0.005s
Aug 08, 2026 12:39:42 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 12:39:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 12:39:42 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 12:39:42 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 12:39:43 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 00:39:44.030591 1856 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.208.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.208.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangerkms_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.208.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/rangerkms_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.208.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.208.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.208.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/HTTP_127.1.208.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 00:39:44 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149584, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 00:39:44.112890 1856 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 00:39:44.122954 1856 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 00:39:44.851200 1856 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 00:39:44.851282 1856 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms
I20260808 00:39:44.851298 1856 mini_ranger_kms.cc:227] Using postgres at 127.1.208.20:38509
2026-08-08 00:39:45,044 [I] DB FLAVOR :POSTGRES
2026-08-08 00:39:45,045 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 00:39:45,045 [I] Checking connection
2026-08-08 00:39:45,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 00:39:45,291 [I] connection success
2026-08-08 00:39:45,291 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 00:39:45,291 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 00:39:45,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 00:39:45,531 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 00:39:45,532 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 00:39:45,532 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 00:39:45,532 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.208.20:38509/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 00:39:45,778 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 00:39:46.243881 1856 mini_ranger_kms.cc:310] Created kms service
I20260808 00:39:46.466079 1856 mini_ranger_kms.cc:326] Created kudu user
I20260808 00:39:46.506572 1856 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 00:39:46 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149584, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:46.881132 1856 mini_ranger_kms.cc:384] Added ranger policy
I20260808 00:39:46.881385 1856 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms:/tmp/dist-test-tasknSGkLY/build/release/bin/postgresql.jar:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasknSGkLY/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 00:39:46.881415 1856 mini_ranger_kms.cc:240] Using host: 127.1.208.20
I20260808 00:39:46.885608 1856 mini_ranger_kms.cc:276] Ranger KMS PID: 22764
I20260808 00:39:46.885682 1856 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.208.20:60385
Aug 08, 2026 12:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 12:39:47 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 12:39:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 12:39:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 12:39:48 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 12:39:48 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 00:39:49 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:49.524554 1856 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.412s user 0.004s sys 0.173s
I20260808 00:39:49.524683 1856 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 00:39:49.524741 1856 mini_ranger_kms.cc:401] 127.1.208.20:60385/kms/v1/keys
Aug 08 00:39:49 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
Aug 08 00:39:50 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149589, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.208.20@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
Aug 08 00:39:52 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:39:52.315346 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37557
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37557
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:52.403342 22855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:52.403548 22855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:52.403577 22855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:52.404677 22855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:52.404718 22855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:52.404737 22855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:52.404752 22855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:52.404768 22855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:52.406421 22855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.208.20:60385/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37557
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:52.406661 22855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:52.406862 22855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:52.411970 22861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:52.411948 22860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:52.412004 22855 server_base.cc:1034] running on GCE node
W20260808 00:39:52.411975 22863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:52.412511 22855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:52.412799 22855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:52.413939 22855 hybrid_clock.cc:648] HybridClock initialized: now 1786149592413941 us; error 47 us; skew 500 ppm
Aug 08 00:39:52 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149592, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:52.420477 22855 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:39:52.420961 22855 webserver.cc:460] Webserver started at http://127.1.208.62:44573/ using document root <none> and password file <none>
I20260808 00:39:52.421159 22855 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:52.421200 22855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:52.421360 22855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:52 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149592, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:52.476049 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "85de374da6c54dd6967b5970dc4a822c"
format_stamp: "Formatted at 2026-08-08 00:39:52 on dist-test-slave-gjw7"
server_key: "4c4441c6b99bc9de171a16a8d4c90acc"
server_key_iv: "2762d4b352e90ca18e7b98d39d114a79"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:52.476441 22855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "85de374da6c54dd6967b5970dc4a822c"
format_stamp: "Formatted at 2026-08-08 00:39:52 on dist-test-slave-gjw7"
server_key: "4c4441c6b99bc9de171a16a8d4c90acc"
server_key_iv: "2762d4b352e90ca18e7b98d39d114a79"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:52.478076 22855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:39:52.493230 22877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:52.493476 22855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:39:52.493602 22855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "85de374da6c54dd6967b5970dc4a822c"
format_stamp: "Formatted at 2026-08-08 00:39:52 on dist-test-slave-gjw7"
server_key: "4c4441c6b99bc9de171a16a8d4c90acc"
server_key_iv: "2762d4b352e90ca18e7b98d39d114a79"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:52.493716 22855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:52.508246 22855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:52.510320 22855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:52.510459 22855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:52.515316 22855 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37557
I20260808 00:39:52.515370 22970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37557 every 8 connection(s)
I20260808 00:39:52.515789 22855 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:39:52.516753 22971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:52.520473 22971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Bootstrap starting.
I20260808 00:39:52.521121 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 22855
I20260808 00:39:52.521214 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:39:52.521399 1856 external_mini_cluster.cc:1428] Setting key 666e6bec93b1e3f43d303c82fee320e6
I20260808 00:39:52.521392 22971 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:52.522061 22971 log.cc:824] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:52.523026 22971 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: No bootstrap required, opened a new log
I20260808 00:39:52.525875 22971 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:52.526080 22971 raft_consensus.cc:374] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:52.526117 22971 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85de374da6c54dd6967b5970dc4a822c, State: Initialized, Role: FOLLOWER
I20260808 00:39:52.526298 22971 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:52.526392 22971 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:39:52.526424 22971 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:39:52.526495 22971 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:52.527264 22971 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:52.527381 22971 leader_election.cc:302] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85de374da6c54dd6967b5970dc4a822c; no voters:
I20260808 00:39:52.527559 22971 leader_election.cc:288] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:39:52.527613 22977 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Leader election won for term 1
Aug 08 00:39:52 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:52.527868 22977 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 LEADER]: Becoming Leader. State: Replica: 85de374da6c54dd6967b5970dc4a822c, State: Running, Role: LEADER
I20260808 00:39:52.527973 22971 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:52.528404 22977 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:52.529016 22977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "85de374da6c54dd6967b5970dc4a822c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } } }
I20260808 00:39:52.529166 22977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:52.529390 22977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85de374da6c54dd6967b5970dc4a822c. Latest consensus state: current_term: 1 leader_uuid: "85de374da6c54dd6967b5970dc4a822c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } } }
I20260808 00:39:52.529512 22977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:52.529626 22971 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:39:52.529700 22998 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:39:52.534606 22998 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:39:52.536509 22974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:52.521900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43626 (local address 127.1.208.62:37557)
0808 00:39:52.522110 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:52.522121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:52.522157 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:39:52.522807 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:52.522816 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:52.522874 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:52.523008 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:52.523669 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:52.524576 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:52.525330 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:52.525792 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:52.525894 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:52.528389 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:52.528393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:52.528394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:52.528406 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:52.532864 (+ 4458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:52.533414 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:52.533418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:52.533418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:52.533688 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:52.533943 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:52.533944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:52.533944 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:52.534691 (+ 747us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:52.535467 (+ 776us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:52.535556 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:39:52.536104 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":92,"threads_started":1}
I20260808 00:39:52.542343 22998 catalog_manager.cc:1347] Generated new cluster ID: 03adc1a6f0a3486aacab8c7b1f5d6d20
I20260808 00:39:52.542400 22998 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:39:52.552449 22998 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:39:52.553323 22998 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:39:52.567169 22998 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Generated new TSK 0
I20260808 00:39:52.567507 22998 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:39:53.288115 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:53.399605 23032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:53.399809 23032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:53.399839 23032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:53.400947 23032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:53.400988 23032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:53.401021 23032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:39:53.402529 23032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:53.403019 23032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:53.403249 23032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:53.408346 23040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:53.409359 23032 server_base.cc:1034] running on GCE node
W20260808 00:39:53.408360 23041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:53.408618 23043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:53.409787 23032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:53.410053 23032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:53.411195 23032 hybrid_clock.cc:648] HybridClock initialized: now 1786149593411184 us; error 32 us; skew 500 ppm
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:53.416821 23032 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:39:53.417443 23032 webserver.cc:460] Webserver started at http://127.1.208.1:46295/ using document root <none> and password file <none>
I20260808 00:39:53.417743 23032 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:53.417820 23032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:53.417964 23032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:53.432287 23032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "fd0dc57c483045c38a44c6a6059376f7"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "9fc2deb996e91d276d6e09938ca7c566"
server_key_iv: "8148b1fc29384edea637e51d034dae9f"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.432719 23032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "fd0dc57c483045c38a44c6a6059376f7"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "9fc2deb996e91d276d6e09938ca7c566"
server_key_iv: "8148b1fc29384edea637e51d034dae9f"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.435238 23032 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 00:39:53.450361 23058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.450544 23032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:39:53.450631 23032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "fd0dc57c483045c38a44c6a6059376f7"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "9fc2deb996e91d276d6e09938ca7c566"
server_key_iv: "8148b1fc29384edea637e51d034dae9f"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.450721 23032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:53.467617 23032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:53.469578 23032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:53.469709 23032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:53.469913 23032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:53.470296 23032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:53.470337 23032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.470369 23032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:53.470408 23032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.478498 23032 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35901
I20260808 00:39:53.478607 23229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35901 every 8 connection(s)
I20260808 00:39:53.478914 23032 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:53.486310 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 23032
I20260808 00:39:53.486405 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:39:53.486595 1856 external_mini_cluster.cc:1428] Setting key b5e8f493bcc3370d474423b9a68def4c
I20260808 00:39:53.488273 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.479730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.1:52821)
0808 00:39:53.479904 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:53.479944 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:53.480470 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:53.480835 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:53.480839 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:53.481022 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:53.481274 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.481279 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.482010 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.482011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.482795 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.482798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.483155 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.483156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.483280 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.483747 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:53.483756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:53.484975 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:53.486667 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.486675 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.486677 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.486972 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.487178 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.487178 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.487179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.487317 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.487605 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:53.487612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:53.487788 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 00:39:53.487884 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 00:39:53.488034 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260808 00:39:53.488583 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.479749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:52821 (local address 127.1.208.62:37557)
0808 00:39:53.480624 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:53.480626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:53.480632 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:39:53.480656 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:53.480657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:53.480667 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:53.480728 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:53.481388 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.481931 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.482879 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.483074 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.483130 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.485060 (+ 1930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:53.485061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:53.485061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:53.485069 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:53.486575 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.487052 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.487053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.487053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.487116 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.487388 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.487389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.487389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.487522 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:53.487580 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:53.488415 (+ 835us) server_negotiation.cc:299] Negotiation successful
0808 00:39:53.488494 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":846,"thread_start_us":75,"threads_started":1}
I20260808 00:39:53.490132 23230 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:53.490240 23230 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:53.490424 23230 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:53.491005 22908 ts_manager.cc:194] Registered new tserver with Master: fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901)
I20260808 00:39:53.491593 22908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:52821
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:39:53.514487 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:53.642860 23238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:53.643102 23238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:53.643134 23238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:53.644318 23238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:53.644419 23238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:53.644469 23238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:53.646073 23238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:53.646307 23238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:53.646574 23238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:53.652562 23252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:53.652587 23249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:53.653007 23250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:53.653430 23238 server_base.cc:1034] running on GCE node
I20260808 00:39:53.653672 23238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:53.653944 23238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:53.655088 23238 hybrid_clock.cc:648] HybridClock initialized: now 1786149593655066 us; error 35 us; skew 500 ppm
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:53.658468 23238 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:39:53.659070 23238 webserver.cc:460] Webserver started at http://127.1.208.2:37693/ using document root <none> and password file <none>
I20260808 00:39:53.659384 23238 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:53.659518 23238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:53.659711 23238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:53.674911 23238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "2a305fc8026a4853928c62093d0f4d41"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "207cc3dc5ead72ba59c0f7bf4fd94503"
server_key_iv: "64a270fccb5801f0bdedf01c48019091"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.675331 23238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "2a305fc8026a4853928c62093d0f4d41"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "207cc3dc5ead72ba59c0f7bf4fd94503"
server_key_iv: "64a270fccb5801f0bdedf01c48019091"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.679805 23238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260808 00:39:53.702744 23265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.702996 23238 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260808 00:39:53.703112 23238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "2a305fc8026a4853928c62093d0f4d41"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "207cc3dc5ead72ba59c0f7bf4fd94503"
server_key_iv: "64a270fccb5801f0bdedf01c48019091"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.703208 23238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:53.719345 23238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:53.721884 23238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:53.722077 23238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:53.722349 23238 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:53.722710 23238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:53.722749 23238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.722779 23238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:53.722842 23238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.730880 23238 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:43729
I20260808 00:39:53.731257 23238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:39:53.731818 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 23238
I20260808 00:39:53.731941 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:39:53.732175 1856 external_mini_cluster.cc:1428] Setting key 0a56e9f67487589073eadd9565f36f29
I20260808 00:39:53.732158 23437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:43729 every 8 connection(s)
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:53.742302 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.732034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.2:48273)
0808 00:39:53.732520 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:53.732572 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:53.733214 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:53.733597 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:53.733602 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:53.733854 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:53.734127 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.734134 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.735056 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.735056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.735882 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.735885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.736435 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.736436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.736580 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.737122 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:53.737133 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:53.738740 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:53.740578 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.740586 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.740589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.740898 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.741116 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.741116 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.741117 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.741253 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.741584 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:53.741591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:53.741778 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 00:39:53.741894 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 00:39:53.742058 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":100,"threads_started":1}
I20260808 00:39:53.742864 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.732425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:48273 (local address 127.1.208.62:37557)
0808 00:39:53.732476 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:53.732478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:53.732647 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:39:53.733373 (+ 726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:53.733374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:53.733386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:53.733414 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:53.734244 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.734933 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.735976 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.736284 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.736345 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.738851 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:53.738853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:53.738853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:53.738860 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:53.740487 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.740983 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.740985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.740985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.741049 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.741344 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.741346 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.741346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.741492 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:53.741556 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:53.742670 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0808 00:39:53.742763 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:39:53.743563 23438 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:53.743666 23438 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:53.743853 23438 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:53.744218 22908 ts_manager.cc:194] Registered new tserver with Master: 2a305fc8026a4853928c62093d0f4d41 (127.1.208.2:43729)
I20260808 00:39:53.744700 22908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:48273
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:39:53.764616 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:53.858004 23445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:53.858210 23445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:53.858240 23445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:53.859395 23445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:53.859433 23445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:53.859467 23445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:39:53.860998 23445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:53.861245 23445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:53.861506 23445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:53.866556 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:53.866636 23456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:53.866984 23457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:53.867311 23445 server_base.cc:1034] running on GCE node
I20260808 00:39:53.867527 23445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:53.867753 23445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:53.868897 23445 hybrid_clock.cc:648] HybridClock initialized: now 1786149593868891 us; error 32 us; skew 500 ppm
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:53.871598 23445 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:39:53.872071 23445 webserver.cc:460] Webserver started at http://127.1.208.3:46263/ using document root <none> and password file <none>
I20260808 00:39:53.872303 23445 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:53.872354 23445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:53.872489 23445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:53.897981 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "02fc8ac2689f445f8b40903adc1210d9"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "183b7370a6b7210d1448234db035fa34"
server_key_iv: "0e0d519fc29108c2063df81cf7fdeafa"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.898677 23445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "02fc8ac2689f445f8b40903adc1210d9"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "183b7370a6b7210d1448234db035fa34"
server_key_iv: "0e0d519fc29108c2063df81cf7fdeafa"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.900899 23445 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 00:39:53.917927 23470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.919100 23445 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 00:39:53.919226 23445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "02fc8ac2689f445f8b40903adc1210d9"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "183b7370a6b7210d1448234db035fa34"
server_key_iv: "0e0d519fc29108c2063df81cf7fdeafa"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:53.919339 23445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:53.957870 23445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:53.960218 23445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:53.960345 23445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:53.960553 23445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:53.960940 23445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:53.960989 23445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.961020 23445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:53.961063 23445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:53.976958 23445 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:41833
I20260808 00:39:53.977430 23445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:39:53.980031 23652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:41833 every 8 connection(s)
I20260808 00:39:53.984825 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 23445
I20260808 00:39:53.984946 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:39:53.985148 1856 external_mini_cluster.cc:1428] Setting key 3211595a8c9d0b273e6209679a1fd01e
Aug 08 00:39:53 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149593, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:53.989866 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.980497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:40057 (local address 127.1.208.62:37557)
0808 00:39:53.980560 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:53.980562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:53.980867 (+ 305us) server_negotiation.cc:407] Connection header received
0808 00:39:53.981395 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:53.981397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:53.981411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:53.981482 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:53.982114 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.982858 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.983872 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.984149 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.984220 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.987054 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:53.987056 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:53.987056 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:53.987064 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:53.988542 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.988908 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.988909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.988910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.988977 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:53.989207 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:53.989208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:53.989209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:53.989368 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:53.989444 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:53.989686 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 00:39:53.989766 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:39:53.990006 23657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:53.980424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.3:40057)
0808 00:39:53.980727 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:53.980773 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:53.981274 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:53.981555 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:53.981559 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:53.981755 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:53.982018 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.982024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.982946 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.982947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.983785 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:53.983788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:53.984389 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:53.984390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:53.984536 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:53.985323 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:53.985334 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:53.986970 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:53.988616 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.988622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.988624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.988837 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.989039 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:53.989039 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:53.989040 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:53.989148 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:53.989433 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:53.989437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:53.989562 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 00:39:53.989646 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:39:53.989770 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":141,"threads_started":1}
I20260808 00:39:53.990545 23655 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:53.990657 23655 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:53.990867 23655 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:53.991259 22908 ts_manager.cc:194] Registered new tserver with Master: 02fc8ac2689f445f8b40903adc1210d9 (127.1.208.3:41833)
I20260808 00:39:53.991739 22908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:40057
I20260808 00:39:53.997545 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:39:53.998255 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 23032
I20260808 00:39:54.025825 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 23238
I20260808 00:39:54.037781 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 23445
I20260808 00:39:54.062824 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 22855
I20260808 00:39:54.074568 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37557
--webserver_interface=127.1.208.62
--webserver_port=44573
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:37557
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:54.155886 23661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:54.156105 23661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:54.156124 23661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:54.157217 23661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:54.157251 23661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:54.157264 23661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:54.157275 23661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:54.157310 23661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:54.158874 23661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:37557
--ranger_config_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:37557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=44573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:54.159071 23661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:54.159274 23661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:39:54.164057 23661 server_base.cc:1034] running on GCE node
W20260808 00:39:54.163993 23672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.163993 23676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.164026 23679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.164515 23661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:54.164757 23661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:54.165885 23661 hybrid_clock.cc:648] HybridClock initialized: now 1786149594165875 us; error 30 us; skew 500 ppm
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:54.168308 23661 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:39:54.168731 23661 webserver.cc:460] Webserver started at http://127.1.208.62:44573/ using document root <none> and password file <none>
I20260808 00:39:54.168929 23661 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:54.168979 23661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:54.204674 23661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:54.205648 23692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.205842 23661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:54.205937 23661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "85de374da6c54dd6967b5970dc4a822c"
format_stamp: "Formatted at 2026-08-08 00:39:52 on dist-test-slave-gjw7"
server_key: "4c4441c6b99bc9de171a16a8d4c90acc"
server_key_iv: "2762d4b352e90ca18e7b98d39d114a79"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:54.206332 23661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:54.218626 23661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:54.220456 23661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:54.220580 23661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:54.224650 23661 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:37557
I20260808 00:39:54.224686 23781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:37557 every 8 connection(s)
I20260808 00:39:54.225051 23661 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:39:54.226333 23783 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:39:54.226931 23783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Bootstrap starting.
I20260808 00:39:54.230157 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 23661
I20260808 00:39:54.230412 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:35901
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:37557
--webserver_port=46295
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:39:54.234754 23783 log.cc:824] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:54.236399 23783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:39:54.236637 23783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Bootstrap complete.
I20260808 00:39:54.238806 23783 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:54.239040 23783 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85de374da6c54dd6967b5970dc4a822c, State: Initialized, Role: FOLLOWER
I20260808 00:39:54.239199 23783 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:54.239274 23783 raft_consensus.cc:388] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:39:54.239332 23783 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:39:54.239380 23783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:39:54.240381 23783 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:54.240471 23783 leader_election.cc:302] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 85de374da6c54dd6967b5970dc4a822c; no voters:
I20260808 00:39:54.240625 23783 leader_election.cc:288] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:39:54.240674 23788 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:39:54.240849 23788 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [term 2 LEADER]: Becoming Leader. State: Replica: 85de374da6c54dd6967b5970dc4a822c, State: Running, Role: LEADER
I20260808 00:39:54.240900 23783 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:54.241031 23788 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } }
I20260808 00:39:54.241453 23790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85de374da6c54dd6967b5970dc4a822c. Latest consensus state: current_term: 2 leader_uuid: "85de374da6c54dd6967b5970dc4a822c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } } }
I20260808 00:39:54.241461 23789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "85de374da6c54dd6967b5970dc4a822c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "85de374da6c54dd6967b5970dc4a822c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 37557 } } }
I20260808 00:39:54.241552 23790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:54.241570 23789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:54.241914 23802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:39:54.242414 23802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:39:54.242596 23802 catalog_manager.cc:1259] Loaded cluster ID: 03adc1a6f0a3486aacab8c7b1f5d6d20
I20260808 00:39:54.242668 23802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:39:54.242774 23783 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 00:39:54.243901 23802 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:39:54.244081 23802 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Loaded TSK: 0
I20260808 00:39:54.244295 23802 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:39:54.345628 23786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:54.345917 23786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:54.345957 23786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:54.347648 23786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:54.347743 23786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:54.347801 23786 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:39:54.350039 23786 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:35901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=46295
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23786
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:54.350445 23786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:54.350764 23786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:54.357564 23843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.357570 23841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.357964 23786 server_base.cc:1034] running on GCE node
W20260808 00:39:54.357921 23845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.358299 23786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:54.358572 23786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:54.359735 23786 hybrid_clock.cc:648] HybridClock initialized: now 1786149594359706 us; error 41 us; skew 500 ppm
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:54.362970 23786 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:39:54.363504 23786 webserver.cc:460] Webserver started at http://127.1.208.1:46295/ using document root <none> and password file <none>
I20260808 00:39:54.363817 23786 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:54.363946 23786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:54.384338 23786 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:54.390021 23859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.390273 23786 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260808 00:39:54.390378 23786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "fd0dc57c483045c38a44c6a6059376f7"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "9fc2deb996e91d276d6e09938ca7c566"
server_key_iv: "8148b1fc29384edea637e51d034dae9f"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:54.390904 23786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:54.410419 23786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:54.412582 23786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:54.412814 23786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:54.413137 23786 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:54.413605 23786 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:54.413700 23786 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.413827 23786 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:54.413908 23786 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.420604 23786 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35901
I20260808 00:39:54.421046 24045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35901 every 8 connection(s)
I20260808 00:39:54.421113 23786 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:39:54.427059 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 23786
I20260808 00:39:54.427368 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:43729
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:37557
--webserver_port=37693
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:54.434437 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.421958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.1:55397)
0808 00:39:54.422143 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:54.422193 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:54.422837 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:54.423393 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:54.423398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:54.423630 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:54.423923 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.423930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.425661 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.425662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.426937 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.426940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.427889 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.427890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.428033 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.428562 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:54.428572 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:54.430074 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:54.432286 (+ 2212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.432293 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.432295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.432584 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.432929 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.432930 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.432930 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.433146 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.433690 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:54.433696 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:54.433869 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 00:39:54.433976 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:39:54.434136 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260808 00:39:54.435086 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.422042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:55397 (local address 127.1.208.62:37557)
0808 00:39:54.422270 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:54.422280 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:54.422324 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:39:54.423099 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:54.423107 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:54.423174 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:54.423325 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:54.424051 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.425533 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.427036 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.427655 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.427758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.430197 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:54.430201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:54.430202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:54.430215 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:54.432194 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.432677 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.432681 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.432681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.432817 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.433227 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.433228 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.433229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.433577 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:54.433658 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:54.434784 (+ 1126us) server_negotiation.cc:299] Negotiation successful
0808 00:39:54.434903 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":89,"threads_started":1}
I20260808 00:39:54.435738 24050 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:54.435844 24050 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.436048 24050 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.437000 24050 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37557 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:39:54.437067 24050 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.437126 24050 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:54.437568 24050 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.437637 24050 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.437954 24050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:39:54.540194 24054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:54.540390 24054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:54.540421 24054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:54.541602 24054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:54.541643 24054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:54.541677 24054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:39:54.543181 24054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:43729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=37693
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:54.543409 24054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:54.543609 24054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:54.548465 24069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.548538 24066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.548465 24063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.548997 24054 server_base.cc:1034] running on GCE node
I20260808 00:39:54.549194 24054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:54.549454 24054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:54.550630 24054 hybrid_clock.cc:648] HybridClock initialized: now 1786149594550636 us; error 42 us; skew 500 ppm
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:54.553195 24054 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:39:54.553681 24054 webserver.cc:460] Webserver started at http://127.1.208.2:37693/ using document root <none> and password file <none>
I20260808 00:39:54.553887 24054 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:54.553953 24054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:54.574213 24054 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260808 00:39:54.575412 24079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.575646 24054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:54.575766 24054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "2a305fc8026a4853928c62093d0f4d41"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "207cc3dc5ead72ba59c0f7bf4fd94503"
server_key_iv: "64a270fccb5801f0bdedf01c48019091"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:54.576308 24054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:54.595183 24054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:54.597584 24054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:54.597795 24054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:54.598115 24054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:54.598546 24054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:54.598600 24054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.598634 24054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:54.598650 24054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.606139 24054 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:43729
I20260808 00:39:54.606225 24262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:43729 every 8 connection(s)
I20260808 00:39:54.606684 24054 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:39:54.610946 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 24054
I20260808 00:39:54.611253 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.208.20:60385/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:41833
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:37557
--webserver_port=46263
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38355
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:54.617605 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.607437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:43883 (local address 127.1.208.62:37557)
0808 00:39:54.607487 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:54.607489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:54.607721 (+ 232us) server_negotiation.cc:407] Connection header received
0808 00:39:54.608291 (+ 570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:54.608293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:54.608310 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:54.608362 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:54.609008 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.609664 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.610628 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.610898 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.610966 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.614333 (+ 3367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:54.614335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:54.614335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:54.614345 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:54.615888 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.616432 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.616433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.616434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.616501 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.616805 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.616806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.616807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.616952 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:54.617024 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:54.617416 (+ 392us) server_negotiation.cc:299] Negotiation successful
0808 00:39:54.617509 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:39:54.617805 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.607439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.2:43883)
0808 00:39:54.607598 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:54.607639 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:54.608150 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:54.608492 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:54.608497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:54.608700 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:54.608902 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.608907 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.609752 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.609753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.610539 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.610541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.611061 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.611063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.611206 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.611742 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:54.611753 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:54.614237 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:54.615997 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.616005 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.616008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.616319 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.616575 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.616576 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.616576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.616721 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.617020 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:54.617028 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:54.617201 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 00:39:54.617392 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 00:39:54.617565 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260808 00:39:54.618305 24263 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:54.618415 24263 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.618620 24263 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.619421 24263 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37557 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:39:54.619499 24263 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.619570 24263 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:54.619978 24263 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.620054 24263 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.620441 24263 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:39:54.709607 24266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:54.709955 24266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:54.710027 24266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:54.711654 24266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:54.711759 24266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:54.711825 24266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:39:54.713986 24266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38355
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.208.20:60385/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:41833
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=46263
--tserver_master_addrs=127.1.208.62:37557
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:54.714362 24266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:54.714660 24266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:54.725378 24277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.725654 24266 server_base.cc:1034] running on GCE node
W20260808 00:39:54.725433 24273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:54.725494 24280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:54.725980 24266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:54.726218 24266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:54.727370 24266 hybrid_clock.cc:648] HybridClock initialized: now 1786149594727356 us; error 25 us; skew 500 ppm
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:54.730233 24266 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:39:54.730660 24266 webserver.cc:460] Webserver started at http://127.1.208.3:46263/ using document root <none> and password file <none>
I20260808 00:39:54.730880 24266 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:54.730959 24266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:54.750573 24266 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:39:54.751434 24289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.751636 24266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:39:54.751731 24266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "02fc8ac2689f445f8b40903adc1210d9"
format_stamp: "Formatted at 2026-08-08 00:39:53 on dist-test-slave-gjw7"
server_key: "183b7370a6b7210d1448234db035fa34"
server_key_iv: "0e0d519fc29108c2063df81cf7fdeafa"
server_key_version: "kuduclusterkey@0"
I20260808 00:39:54.752103 24266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:54.762509 24266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:54.764271 24266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:54.764438 24266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:54.764663 24266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:39:54.765007 24266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:39:54.765066 24266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.765097 24266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:39:54.765161 24266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:54.772274 24266 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:41833
I20260808 00:39:54.772822 24266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:39:54.773264 24475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:41833 every 8 connection(s)
Aug 08 00:39:54 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149594, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:54.779044 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 24266
I20260808 00:39:54.782816 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.774044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:41087 (local address 127.1.208.62:37557)
0808 00:39:54.774097 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:54.774099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:54.774312 (+ 213us) server_negotiation.cc:407] Connection header received
0808 00:39:54.774835 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:54.774836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:54.774850 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:54.774901 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:54.775437 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.775947 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.776914 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.777114 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.777171 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.779723 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:54.779725 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:54.779725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:54.779733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:54.781336 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.781808 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.781810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.781810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.781877 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.782132 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.782133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.782133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.782252 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:54.782293 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:54.782646 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 00:39:54.782729 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:39:54.782980 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.774046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:37557 (local address 127.1.208.3:41087)
0808 00:39:54.774194 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:54.774235 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:54.774735 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:54.774972 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:54.774976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:54.775156 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:54.775345 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.775350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.776033 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.776034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.776821 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:54.776824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.777203 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.777205 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:54.777347 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.777837 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:54.777846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:54.779613 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:54.781424 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.781431 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.781433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.781719 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.781945 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:54.781945 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:54.781946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:54.782067 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:54.782356 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:54.782363 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:54.782514 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 00:39:54.782612 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 00:39:54.782761 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":81,"threads_started":1}
I20260808 00:39:54.783440 24476 heartbeater.cc:347] Connected to a master server at 127.1.208.62:37557
I20260808 00:39:54.783535 24476 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.783703 24476 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.784363 24476 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:37557 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 00:39:54.784436 24476 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.784500 24476 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:54.784873 24476 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:54.784943 24476 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
W20260808 00:39:54.785279 24476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:39:54.787797 24481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:54.779439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43640 (local address 127.1.208.62:37557)
0808 00:39:54.779573 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:54.779575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:54.779692 (+ 117us) server_negotiation.cc:407] Connection header received
0808 00:39:54.779794 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:54.779795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:54.779806 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:54.779844 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:54.780628 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.781285 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.783351 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:54.783654 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:54.783731 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:54.785385 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:54.785386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:54.785387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:54.785393 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:54.786749 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.787051 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.787052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.787052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.787101 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:54.787260 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:54.787262 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:54.787262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:54.787362 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:54.787514 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:54.787549 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 00:39:54.787623 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":73,"threads_started":1}
Aug 08 00:39:55 dist-test-slave-gjw7 krb5kdc[22102](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786149595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:55.438158 24050 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:55.438562 24050 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:55.439563 23724 ts_manager.cc:194] Registered new tserver with Master: fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901)
I20260808 00:39:55.441640 23724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:55397
I20260808 00:39:55.620805 24263 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:55.621605 24263 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:55.622792 23724 ts_manager.cc:194] Registered new tserver with Master: 2a305fc8026a4853928c62093d0f4d41 (127.1.208.2:43729)
I20260808 00:39:55.624256 23724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:43883
I20260808 00:39:55.785636 24476 heartbeater.cc:464] Registering TS with master...
I20260808 00:39:55.786358 24476 heartbeater.cc:511] Master 127.1.208.62:37557 requested a full tablet report, sending...
I20260808 00:39:55.787166 23724 ts_manager.cc:194] Registered new tserver with Master: 02fc8ac2689f445f8b40903adc1210d9 (127.1.208.3:41833)
I20260808 00:39:55.788291 23724 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:41087
I20260808 00:39:55.793762 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:39:55.810316 24511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.797389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.1.208.62:37557)
0808 00:39:55.797620 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:55.797628 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:55.797687 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:39:55.797758 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:55.797759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:55.797769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:55.797842 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:55.798439 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.798977 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.801610 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.802012 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.802081 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.803876 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:55.803877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:55.803878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:55.803886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:55.808917 (+ 5031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:55.809375 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:55.809377 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:55.809377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:55.809455 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:55.809648 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:55.809649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:55.809650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:55.809796 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:55.809892 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:55.810041 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 00:39:55.810122 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
I20260808 00:39:55.812737 23724 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:39:55.817598 23724 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:39:55.836954 24519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.830960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43729 (local address 127.0.0.1:56354)
0808 00:39:55.831785 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:55.832433 (+ 648us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:55.832476 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:55.832841 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:55.832842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:55.832844 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:55.832909 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.832911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.834131 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.834132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.835517 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.835521 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.836608 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.836610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.836710 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.836725 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:39:55.836796 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:39:55.836853 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":111,"thread_start_us":80,"threads_started":1}
I20260808 00:39:55.837447 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.831552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.1.208.1:35901)
0808 00:39:55.832165 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:55.832170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:55.832178 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:55.832218 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:55.832220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:55.832239 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:55.832309 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:55.832524 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.833783 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.835621 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.836436 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.836501 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.836563 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:55.837251 (+ 688us) server_negotiation.cc:299] Negotiation successful
0808 00:39:55.837351 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":90,"threads_started":1}
I20260808 00:39:55.837180 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.830965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35901 (local address 127.0.0.1:33768)
0808 00:39:55.831927 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:55.831968 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:55.832033 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:55.832329 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:55.832333 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:55.832335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:55.832433 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.832436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.833874 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.833876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.835518 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.835521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.836610 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.836611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.836710 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.836724 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:39:55.837059 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 00:39:55.837104 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":35,"thread_start_us":102,"threads_started":1}
I20260808 00:39:55.838449 23909 tablet_service.cc:1467] Processing CreateTablet for tablet c4d07fc511b3499bb35401aa91f7f73c (DEFAULT_TABLE table=test-table [id=3f7de5629ff7493181d25f9cd9bb8d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:55.838702 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.831349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.1.208.2:43729)
0808 00:39:55.832594 (+ 1245us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:55.832598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:55.832607 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:39:55.832650 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:55.832652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:55.832665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:55.832735 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:55.832986 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.834048 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.835621 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.836428 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.836501 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.836562 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:55.838553 (+ 1991us) server_negotiation.cc:299] Negotiation successful
0808 00:39:55.838616 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":85,"threads_started":1}
I20260808 00:39:55.838933 23909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d07fc511b3499bb35401aa91f7f73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:55.840131 24156 tablet_service.cc:1467] Processing CreateTablet for tablet c4d07fc511b3499bb35401aa91f7f73c (DEFAULT_TABLE table=test-table [id=3f7de5629ff7493181d25f9cd9bb8d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:55.840513 24156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d07fc511b3499bb35401aa91f7f73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:55.842549 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.831207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41833 (local address 127.0.0.1:37414)
0808 00:39:55.837916 (+ 6709us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:55.837926 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:55.837971 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:55.838285 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:55.838286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:55.838288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:55.838320 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.838321 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.839760 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.839761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.840786 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:55.840790 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.842293 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.842294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:55.842354 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.842359 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:55.842438 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:39:55.842478 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6695,"thread_start_us":71,"threads_started":1}
I20260808 00:39:55.843026 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:55.831580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37414 (local address 127.1.208.3:41833)
0808 00:39:55.834446 (+ 2866us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:55.834451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:55.838035 (+ 3584us) server_negotiation.cc:407] Connection header received
0808 00:39:55.838083 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:55.838086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:55.838102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:55.838165 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:55.838398 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.839662 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.840881 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:55.842121 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:55.842189 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:55.842258 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:55.842895 (+ 637us) server_negotiation.cc:299] Negotiation successful
0808 00:39:55.842940 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2820,"thread_start_us":84,"threads_started":1}
I20260808 00:39:55.843444 24528 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Bootstrap starting.
I20260808 00:39:55.843989 24384 tablet_service.cc:1467] Processing CreateTablet for tablet c4d07fc511b3499bb35401aa91f7f73c (DEFAULT_TABLE table=test-table [id=3f7de5629ff7493181d25f9cd9bb8d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:39:55.844321 24528 tablet_bootstrap.cc:654] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:55.844359 24384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4d07fc511b3499bb35401aa91f7f73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:55.844633 24528 log.cc:824] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:55.845726 24529 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Bootstrap starting.
I20260808 00:39:55.846338 24529 tablet_bootstrap.cc:654] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:55.846594 24529 log.cc:824] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:55.847357 24528 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: No bootstrap required, opened a new log
I20260808 00:39:55.847421 24528 ts_tablet_manager.cc:1397] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 00:39:55.848068 24533 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Bootstrap starting.
I20260808 00:39:55.848340 24529 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: No bootstrap required, opened a new log
I20260808 00:39:55.848482 24529 ts_tablet_manager.cc:1397] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:39:55.848760 24533 tablet_bootstrap.cc:654] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:55.849112 24533 log.cc:824] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:55.849867 24533 tablet_bootstrap.cc:492] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: No bootstrap required, opened a new log
I20260808 00:39:55.849931 24533 ts_tablet_manager.cc:1397] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260808 00:39:55.850082 24528 raft_consensus.cc:348] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.850240 24528 raft_consensus.cc:374] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:55.850271 24528 raft_consensus.cc:729] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd0dc57c483045c38a44c6a6059376f7, State: Initialized, Role: FOLLOWER
I20260808 00:39:55.850389 24528 consensus_queue.cc:260] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.850642 24528 ts_tablet_manager.cc:1428] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 00:39:55.850804 24050 heartbeater.cc:503] Master 127.1.208.62:37557 was elected leader, sending a full tablet report...
I20260808 00:39:55.851665 24529 raft_consensus.cc:348] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.851958 24529 raft_consensus.cc:374] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:55.852061 24529 raft_consensus.cc:729] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a305fc8026a4853928c62093d0f4d41, State: Initialized, Role: FOLLOWER
I20260808 00:39:55.852186 24529 consensus_queue.cc:260] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.852334 24533 raft_consensus.cc:348] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.852443 24263 heartbeater.cc:503] Master 127.1.208.62:37557 was elected leader, sending a full tablet report...
I20260808 00:39:55.852494 24533 raft_consensus.cc:374] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:55.852532 24533 raft_consensus.cc:729] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02fc8ac2689f445f8b40903adc1210d9, State: Initialized, Role: FOLLOWER
I20260808 00:39:55.852435 24529 ts_tablet_manager.cc:1428] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260808 00:39:55.852634 24533 consensus_queue.cc:260] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:55.852902 24476 heartbeater.cc:503] Master 127.1.208.62:37557 was elected leader, sending a full tablet report...
I20260808 00:39:55.853248 24533 ts_tablet_manager.cc:1428] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
W20260808 00:39:55.858151 24264 tablet.cc:2369] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:39:55.923522 24051 tablet.cc:2369] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:39:56.003023 24542 raft_consensus.cc:482] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:39:56.003186 24542 raft_consensus.cc:504] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:56.004679 24542 leader_election.cc:288] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901), 02fc8ac2689f445f8b40903adc1210d9 (127.1.208.3:41833)
I20260808 00:39:56.009091 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.004450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:51417 (local address 127.1.208.1:35901)
0808 00:39:56.004510 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:56.004512 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:56.005241 (+ 729us) server_negotiation.cc:407] Connection header received
0808 00:39:56.005396 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:56.005398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:56.005413 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:56.005444 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:56.005766 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.006550 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.008023 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.008599 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.008650 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.008661 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:56.008956 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 00:39:56.009000 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:39:56.009404 24547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.004418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35901 (local address 127.1.208.2:51417)
0808 00:39:56.005156 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.005168 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:56.005274 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:56.005522 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:56.005523 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:56.005525 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:56.005671 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.005675 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.006666 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.006667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.007912 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.007915 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.008757 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.008759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.008866 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.008871 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:56.009241 (+ 370us) client_negotiation.cc:240] Negotiation successful
0808 00:39:56.009284 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":183,"threads_started":1}
I20260808 00:39:56.010159 24546 raft_consensus.cc:482] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:39:56.010272 24546 raft_consensus.cc:504] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:56.010771 24546 leader_election.cc:288] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a305fc8026a4853928c62093d0f4d41 (127.1.208.2:43729), fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901)
I20260808 00:39:56.011371 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "2a305fc8026a4853928c62093d0f4d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd0dc57c483045c38a44c6a6059376f7" is_pre_election: true
I20260808 00:39:56.011579 23960 raft_consensus.cc:2454] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a305fc8026a4853928c62093d0f4d41 in term 0.
I20260808 00:39:56.011889 24081 leader_election.cc:302] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a305fc8026a4853928c62093d0f4d41, fd0dc57c483045c38a44c6a6059376f7; no voters:
I20260808 00:39:56.012048 24542 raft_consensus.cc:2781] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:39:56.012130 24542 raft_consensus.cc:482] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:39:56.012151 24542 raft_consensus.cc:3037] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:56.013204 24542 raft_consensus.cc:504] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:56.013384 24542 leader_election.cc:288] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [CANDIDATE]: Term 1 election: Requested vote from peers fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901), 02fc8ac2689f445f8b40903adc1210d9 (127.1.208.3:41833)
I20260808 00:39:56.013631 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "2a305fc8026a4853928c62093d0f4d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd0dc57c483045c38a44c6a6059376f7"
I20260808 00:39:56.013702 24525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.004972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:38671 (local address 127.1.208.3:41833)
0808 00:39:56.005132 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:56.005134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:56.006110 (+ 976us) server_negotiation.cc:407] Connection header received
0808 00:39:56.006207 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:56.006208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:56.006220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:56.006247 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:56.006610 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.007329 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.009041 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.009688 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.009731 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.009740 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:56.013597 (+ 3857us) server_negotiation.cc:299] Negotiation successful
0808 00:39:56.013641 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 00:39:56.013720 23960 raft_consensus.cc:3037] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:56.013895 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.004844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:41833 (local address 127.1.208.2:38671)
0808 00:39:56.006034 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.006046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:56.006138 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:56.006305 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:56.006306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:56.006307 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:56.006524 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.006526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.007421 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.007422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.008941 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.008944 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.011867 (+ 2923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.011868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.013515 (+ 1647us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.013522 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 00:39:56.013767 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 00:39:56.013807 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1146,"thread_start_us":224,"threads_started":1}
I20260808 00:39:56.014748 23960 raft_consensus.cc:2454] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a305fc8026a4853928c62093d0f4d41 in term 1.
I20260808 00:39:56.015040 24081 leader_election.cc:302] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a305fc8026a4853928c62093d0f4d41, fd0dc57c483045c38a44c6a6059376f7; no voters:
I20260808 00:39:56.015122 24542 raft_consensus.cc:2781] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:39:56.015384 24542 raft_consensus.cc:686] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 1 LEADER]: Becoming Leader. State: Replica: 2a305fc8026a4853928c62093d0f4d41, State: Running, Role: LEADER
I20260808 00:39:56.015487 24542 consensus_queue.cc:237] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } }
I20260808 00:39:56.016469 23724 catalog_manager.cc:5495] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a305fc8026a4853928c62093d0f4d41 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "2a305fc8026a4853928c62093d0f4d41" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a305fc8026a4853928c62093d0f4d41" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 } health_report { overall_health: UNKNOWN } } }
I20260808 00:39:56.017901 24398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "2a305fc8026a4853928c62093d0f4d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02fc8ac2689f445f8b40903adc1210d9" is_pre_election: true
I20260808 00:39:56.018006 24398 raft_consensus.cc:2454] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a305fc8026a4853928c62093d0f4d41 in term 0.
I20260808 00:39:56.018103 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.011124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:35277 (local address 127.1.208.2:43729)
0808 00:39:56.011160 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:56.011162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:56.014159 (+ 2997us) server_negotiation.cc:407] Connection header received
0808 00:39:56.014243 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:56.014244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:56.014252 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:56.014277 (+ 25us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:56.014586 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.015300 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.016935 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.017555 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.017594 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.017603 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:56.018012 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 00:39:56.018049 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:39:56.018152 24400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "2a305fc8026a4853928c62093d0f4d41" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02fc8ac2689f445f8b40903adc1210d9"
I20260808 00:39:56.018277 24400 raft_consensus.cc:3037] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:56.018466 24552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.010902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43729 (local address 127.1.208.3:35277)
0808 00:39:56.014098 (+ 3196us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.014108 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:56.014183 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:56.014325 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:56.014326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:56.014327 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:56.014505 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.014507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.015391 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.015392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.016834 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.016837 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.017798 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.017800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.017927 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.017932 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:39:56.018172 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 00:39:56.018213 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3177,"thread_start_us":130,"threads_started":1}
I20260808 00:39:56.019224 24400 raft_consensus.cc:2454] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a305fc8026a4853928c62093d0f4d41 in term 1.
I20260808 00:39:56.019904 24185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "02fc8ac2689f445f8b40903adc1210d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a305fc8026a4853928c62093d0f4d41" is_pre_election: true
I20260808 00:39:56.020879 24523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.011773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:54151 (local address 127.1.208.1:35901)
0808 00:39:56.012052 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:56.012054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:56.017752 (+ 5698us) server_negotiation.cc:407] Connection header received
0808 00:39:56.017848 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:56.017849 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:56.017858 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:56.017887 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:39:56.018072 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.018720 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.020006 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.020539 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.020579 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.020588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:56.020785 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 00:39:56.020818 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258}
I20260808 00:39:56.021065 24553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.011018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35901 (local address 127.1.208.3:54151)
0808 00:39:56.017691 (+ 6673us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.017700 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:56.017778 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:56.017943 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:56.017944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:56.017946 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:39:56.018002 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.018003 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.018805 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.018806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.019913 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:56.019916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.020644 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.020645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:56.020716 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.020719 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:39:56.020954 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 00:39:56.020992 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6654,"thread_start_us":144,"threads_started":1}
I20260808 00:39:56.021353 23960 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4d07fc511b3499bb35401aa91f7f73c" candidate_uuid: "02fc8ac2689f445f8b40903adc1210d9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd0dc57c483045c38a44c6a6059376f7" is_pre_election: true
I20260808 00:39:56.021454 23960 raft_consensus.cc:2379] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02fc8ac2689f445f8b40903adc1210d9 in current term 1: Already voted for candidate 2a305fc8026a4853928c62093d0f4d41 in this term.
I20260808 00:39:56.021701 24291 leader_election.cc:302] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02fc8ac2689f445f8b40903adc1210d9; no voters: 2a305fc8026a4853928c62093d0f4d41, fd0dc57c483045c38a44c6a6059376f7
I20260808 00:39:56.021867 24546 raft_consensus.cc:2726] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260808 00:39:56.025043 24477 tablet.cc:2369] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:39:56.069454 24522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.065854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.1.208.2:43729)
0808 00:39:56.065905 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:56.065908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:56.065916 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:56.065965 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:56.065966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:56.065992 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:56.066075 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:39:56.066288 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.067100 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.068263 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:56.068536 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:56.068604 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:56.068822 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:39:56.068967 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:39:56.069063 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:56.069349 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 00:39:56.069371 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:39:56.072269 23960 raft_consensus.cc:1261] T c4d07fc511b3499bb35401aa91f7f73c P fd0dc57c483045c38a44c6a6059376f7 [term 1 FOLLOWER]: Refusing update from remote peer 2a305fc8026a4853928c62093d0f4d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:56.072295 24400 raft_consensus.cc:1261] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 1 FOLLOWER]: Refusing update from remote peer 2a305fc8026a4853928c62093d0f4d41: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:56.072619 24542 consensus_queue.cc:1035] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "02fc8ac2689f445f8b40903adc1210d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 41833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:39:56.072755 24554 consensus_queue.cc:1035] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd0dc57c483045c38a44c6a6059376f7" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 08 00:39:56 dist-test-slave-gjw7 krb5kdc[22102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149595, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for HTTP/127.1.208.20@KRBTEST.COM
I20260808 00:39:56.078284 24560 mvcc.cc:205] Tried to move back new op lower bound from 7316068745507844096 to 7316068745279827968. Current Snapshot: MvccSnapshot[applied={T|T < 7316068745507844096}]
I20260808 00:39:56.079602 24562 mvcc.cc:205] Tried to move back new op lower bound from 7316068745507844096 to 7316068745279827968. Current Snapshot: MvccSnapshot[applied={T|T < 7316068745507844096}]
I20260808 00:39:56.081554 23723 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:39:56.081650 23723 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:39:56.082616 23723 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 85de374da6c54dd6967b5970dc4a822c: Sending DeleteTablet for 3 replicas of tablet c4d07fc511b3499bb35401aa91f7f73c
I20260808 00:39:56.083261 23909 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d07fc511b3499bb35401aa91f7f73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:56 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33768
I20260808 00:39:56.083778 24561 mvcc.cc:205] Tried to move back new op lower bound from 7316068745507844096 to 7316068745279827968. Current Snapshot: MvccSnapshot[applied={T|T < 7316068745507844096}]
I20260808 00:39:56.083921 24384 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d07fc511b3499bb35401aa91f7f73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:56 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:37414
I20260808 00:39:56.085052 24156 tablet_service.cc:1514] Processing DeleteTablet for tablet c4d07fc511b3499bb35401aa91f7f73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:39:56 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:56354
I20260808 00:39:56.085611 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 23786
I20260808 00:39:56.100080 24577 tablet_replica.cc:330] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: stopping tablet replica
I20260808 00:39:56.100243 24577 raft_consensus.cc:2229] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:39:56.100375 24577 raft_consensus.cc:2258] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:39:56.100698 24576 tablet_replica.cc:330] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: stopping tablet replica
I20260808 00:39:56.100843 24576 raft_consensus.cc:2229] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:39:56.100979 24576 raft_consensus.cc:2258] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:39:56.103379 24576 ts_tablet_manager.cc:1905] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:39:56.105854 24576 ts_tablet_manager.cc:1918] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:39:56.105996 24576 log.cc:1196] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/c4d07fc511b3499bb35401aa91f7f73c
I20260808 00:39:56.106326 24577 ts_tablet_manager.cc:1905] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:39:56.106359 24576 ts_tablet_manager.cc:1939] T c4d07fc511b3499bb35401aa91f7f73c P 02fc8ac2689f445f8b40903adc1210d9: Deleting consensus metadata
I20260808 00:39:56.106796 23695 catalog_manager.cc:4841] TS 02fc8ac2689f445f8b40903adc1210d9 (127.1.208.3:41833): tablet c4d07fc511b3499bb35401aa91f7f73c (table test-table [id=3f7de5629ff7493181d25f9cd9bb8d69]) successfully deleted
W20260808 00:39:56.107441 23695 connection.cc:535] server connection from 127.1.208.1:55397 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 00:39:56.107679 23696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:35901 (error 108)
I20260808 00:39:56.108487 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 24054
I20260808 00:39:56.108757 24577 ts_tablet_manager.cc:1918] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:39:56.108829 24577 log.cc:1196] T c4d07fc511b3499bb35401aa91f7f73c P 2a305fc8026a4853928c62093d0f4d41: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/c4d07fc511b3499bb35401aa91f7f73c
I20260808 00:39:56.121069 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.120928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35901 (local address 127.0.0.1:33770)
0808 00:39:56.120984 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.121027 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:39:56.121320 23696 catalog_manager.cc:4584] TS fd0dc57c483045c38a44c6a6059376f7 (127.1.208.1:35901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4d07fc511b3499bb35401aa91f7f73c: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35901: connect: Connection refused (error 111)
I20260808 00:39:56.126128 24521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:56.126012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43729 (local address 127.0.0.1:56368)
0808 00:39:56.126063 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:56.126089 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.2:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:39:56.126151 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 24266
I20260808 00:39:56.138667 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 23661
I20260808 00:39:56.155386 1856 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 00:39:56.501076 1856 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 00:39:56.501173 1856 mini_ranger.cc:66] Stopping Ranger...
I20260808 00:39:56.845650 1856 mini_ranger.cc:68] Stopped Ranger
2026-08-08 00:39:56.845 UTC [22149] LOG: received smart shutdown request
2026-08-08 00:39:56.850 UTC [22149] LOG: background worker "logical replication launcher" (PID 22163) exited with exit code 1
2026-08-08 00:39:56.850 UTC [22155] LOG: shutting down
2026-08-08 00:39:57.000 UTC [22149] LOG: database system is shut down
2026-08-08T00:39:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48865 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:39:57 dist-test-slave-gjw7 krb5kdc[24592](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:39:57 dist-test-slave-gjw7 krb5kdc[24592](info): set up 2 sockets
Aug 08 00:39:57 dist-test-slave-gjw7 krb5kdc[24592](info): commencing operation
krb5kdc: starting...
W20260808 00:39:58.276885 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:39:58.396494 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42841
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:39:58.478825 24609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:58.479038 24609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:58.479058 24609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:58.480178 24609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:58.480219 24609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:58.480232 24609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:58.480245 24609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:58.480268 24609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:58.481798 24609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:42841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:58.482033 24609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:58.482245 24609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:58.486965 24621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:58.487017 24609 server_base.cc:1034] running on GCE node
W20260808 00:39:58.487067 24618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:58.486979 24617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:58.487502 24609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:58.487780 24609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:58.488921 24609 hybrid_clock.cc:648] HybridClock initialized: now 1786149598488909 us; error 31 us; skew 500 ppm
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:58.494217 24609 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:39:58.494668 24609 webserver.cc:460] Webserver started at http://127.1.208.62:46463/ using document root <none> and password file <none>
I20260808 00:39:58.494875 24609 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:58.494920 24609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:58.495046 24609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:39:58.496119 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "8ad71b48ee5f4186842dfa407b844cc5"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "722feb881d6388077795fd66edece185"
server_key_iv: "dc9669f6e16dfd802676856af88c91b3"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.496459 24609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "8ad71b48ee5f4186842dfa407b844cc5"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "722feb881d6388077795fd66edece185"
server_key_iv: "dc9669f6e16dfd802676856af88c91b3"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.497781 24609 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:39:58.498528 24631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:58.498665 24609 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:39:58.498744 24609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "8ad71b48ee5f4186842dfa407b844cc5"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "722feb881d6388077795fd66edece185"
server_key_iv: "dc9669f6e16dfd802676856af88c91b3"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.498812 24609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:58.519897 24609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:58.522176 24609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:58.522305 24609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:58.526388 24609 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:42841
I20260808 00:39:58.526428 24715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:42841 every 8 connection(s)
I20260808 00:39:58.526774 24609 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:39:58.527652 24717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:58.529277 24717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:39:58.531708 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 24609
I20260808 00:39:58.531823 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:39:58.532011 1856 external_mini_cluster.cc:1428] Setting key 5805c1a23749a22d5dbfd74cc7c6cbaf
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:58.539244 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.529997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42841 (local address 127.0.0.1:42068)
0808 00:39:58.530165 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.530204 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.530668 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.530911 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.530913 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.531156 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.531338 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.531342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.532688 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.532689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.533420 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.533422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.534014 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.534016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.534093 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.534653 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.534664 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.536200 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.537874 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.537882 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.537884 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.538071 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.538282 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.538283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.538283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.538360 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.538685 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.538691 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.538755 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.538836 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.539062 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":87,"thread_start_us":70,"threads_started":1}
I20260808 00:39:58.539258 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.530027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42068 (local address 127.1.208.62:42841)
0808 00:39:58.530173 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.530182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.530287 (+ 105us) server_negotiation.cc:407] Connection header received
0808 00:39:58.530789 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.530791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.530807 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.530895 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.531410 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.532591 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.533504 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.533933 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.534015 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.536281 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.536289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.536290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.536296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.537800 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.538145 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.538149 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.538149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.538223 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.538414 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.538415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.538415 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.538624 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.538699 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.538868 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.538988 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260808 00:39:58.540263 24717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.208.61@KRBTEST.COM; defaulting to no policy
I20260808 00:39:58.540580 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.540471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45874)
0808 00:39:58.540513 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.540557 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 00:39:58.540683 24634 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
I20260808 00:39:58.541082 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.540985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45876)
0808 00:39:58.541031 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.541060 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:39:58.541177 24717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:37653: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
I20260808 00:39:58.557968 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:37653
--webserver_interface=127.1.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:39:58.590376 24717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } attempt: 1
I20260808 00:39:58.590852 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.590732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45892)
0808 00:39:58.590787 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.590819 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 00:39:58.591189 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.591109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45898)
0808 00:39:58.591147 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.591170 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 00:39:58.591265 24717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:37653: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
W20260808 00:39:58.639727 24728 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:58.639925 24728 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:58.639956 24728 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:39:58.641049 24728 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:58.641089 24728 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:58.641108 24728 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:58.641124 24728 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:58.641140 24728 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:58.642686 24728 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:37653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24728
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:58.642928 24728 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:58.643131 24728 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:58.647609 24744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:58.647614 24741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:58.647609 24738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:58.647989 24728 server_base.cc:1034] running on GCE node
I20260808 00:39:58.648217 24728 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:58.648450 24728 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:58.649578 24728 hybrid_clock.cc:648] HybridClock initialized: now 1786149598649568 us; error 27 us; skew 500 ppm
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:58.651898 24728 init.cc:377] Logged in from keytab as kudu/127.1.208.61@KRBTEST.COM (short username kudu)
I20260808 00:39:58.652375 24728 webserver.cc:460] Webserver started at http://127.1.208.61:40213/ using document root <none> and password file <none>
I20260808 00:39:58.652570 24728 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:58.652617 24728 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:58.652740 24728 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:39:58.653860 24728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data/instance:
uuid: "7123513f02ed4591bc791322c8804103"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "b1288283ae2fd7760574ab25edd71d48"
server_key_iv: "e0f0da343a22cc6fa9f647dc6dc0d130"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.654203 24728 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal/instance:
uuid: "7123513f02ed4591bc791322c8804103"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "b1288283ae2fd7760574ab25edd71d48"
server_key_iv: "e0f0da343a22cc6fa9f647dc6dc0d130"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.655401 24728 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:39:58.656049 24753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:58.656193 24728 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:39:58.656275 24728 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "7123513f02ed4591bc791322c8804103"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "b1288283ae2fd7760574ab25edd71d48"
server_key_iv: "e0f0da343a22cc6fa9f647dc6dc0d130"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.656335 24728 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:58.659402 24717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } attempt: 2
I20260808 00:39:58.659757 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.659650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45910)
0808 00:39:58.659700 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.659729 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:39:58.660068 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.659983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45914)
0808 00:39:58.660022 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.660044 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 00:39:58.660172 24717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:37653: Network error: Client connection negotiation failed: client connection to 127.1.208.61:37653: connect: Connection refused (error 111)
I20260808 00:39:58.668625 24728 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:58.670320 24728 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:58.670451 24728 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:58.674428 24728 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.61:37653
I20260808 00:39:58.674455 24862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.61:37653 every 8 connection(s)
I20260808 00:39:58.674762 24728 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
I20260808 00:39:58.675582 24864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:58.677027 24864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:58.682698 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 24728
I20260808 00:39:58.682798 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-1/wal/instance
I20260808 00:39:58.683001 1856 external_mini_cluster.cc:1428] Setting key 9b02a8a98405fd5c2f5e810fc7fd3762
I20260808 00:39:58.685117 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.677702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.1.208.62:42841)
0808 00:39:58.677755 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.677757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.677928 (+ 171us) server_negotiation.cc:407] Connection header received
0808 00:39:58.678367 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.678368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.678378 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.678434 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.678989 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.679461 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.680328 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.680514 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.680556 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.682465 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.682467 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.682468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.682475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.683922 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.684263 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.684265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.684265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.684326 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.684518 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.684519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.684519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.684641 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.684696 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.684988 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.685054 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:39:58.685344 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.677653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42841 (local address 127.0.0.1:42084)
0808 00:39:58.677800 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.677852 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.678292 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.678494 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.678497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.678730 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.678927 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.678932 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.679518 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.679518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.680270 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.680272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.680563 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.680564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.680652 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.681086 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.681095 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.682384 (+ 1289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.683995 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.684002 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.684003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.684200 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.684384 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.684384 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.684384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.684471 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.684695 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.684698 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.684860 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.684969 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.685116 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260808 00:39:58.685878 24864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.208.60@KRBTEST.COM; defaulting to no policy
I20260808 00:39:58.692790 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.686104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45918)
0808 00:39:58.686176 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.686185 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.686226 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.686476 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.686477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.686579 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.686622 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.686623 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.687738 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.687739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.688232 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.688234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.688811 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.688811 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.688866 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.689186 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.689190 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.690267 (+ 1077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.692025 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.692026 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.692027 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.692175 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.692359 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.692359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.692360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.692401 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.692593 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.692593 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.692624 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.692663 (+ 39us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.692731 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 00:39:58.692881 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.686183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45918 (local address 127.1.208.61:37653)
0808 00:39:58.686332 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.686337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.686361 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:39:58.686383 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.686385 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.686397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.686467 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.686681 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.687657 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.688305 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.688739 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.688809 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.690346 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.690355 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.690356 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.690364 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.691935 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.692248 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.692251 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.692252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.692308 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.692434 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.692435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.692435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.692551 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.692605 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.692694 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.692787 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260808 00:39:58.693511 24864 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:39:58.693768 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.693687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59482)
0808 00:39:58.693714 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.693752 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260808 00:39:58.693881 24758 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
I20260808 00:39:58.694267 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.694192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59484)
0808 00:39:58.694227 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.694251 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 00:39:58.694347 24864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38539: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
I20260808 00:39:58.708145 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:38539
--webserver_interface=127.1.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:36971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:39:58.743554 24864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } attempt: 1
I20260808 00:39:58.744016 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.743885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59490)
0808 00:39:58.743950 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.743984 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:39:58.744401 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.744317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59500)
0808 00:39:58.744358 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.744382 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:39:58.744493 24864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38539: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
I20260808 00:39:58.751318 24717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } attempt: 3
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:39:58.758134 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.751677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.1.208.61:37653)
0808 00:39:58.751725 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.751727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.751736 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:39:58.751798 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.751799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.751807 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.751859 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.752211 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.752702 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.753655 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.753863 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.753914 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.755835 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.755837 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.755838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.755845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.757146 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.757522 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.757523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.757523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.757572 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.757744 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.757745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.757745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.757837 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.757878 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.758018 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.758072 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:39:58.758190 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.751634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45926)
0808 00:39:58.751685 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.751696 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.751737 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.751879 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.751880 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.752064 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.752142 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.752144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.752764 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.752765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.753586 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.753589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.753925 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.753926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.754000 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.754412 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.754417 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.755743 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.757225 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.757226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.757226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.757456 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.757627 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.757628 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.757629 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.757690 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.757883 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.757884 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.757931 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.758002 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.758095 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:39:58.758582 24717 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:39:58.758908 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.758805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59510)
0808 00:39:58.758855 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.758886 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 00:39:58.759174 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.759100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59520)
0808 00:39:58.759135 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.759158 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 00:39:58.759250 24717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38539: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
I20260808 00:39:58.790406 24717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } attempt: 1
W20260808 00:39:58.790334 24874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:39:58.790583 24874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:39:58.790616 24874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 00:39:58.790899 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.790744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59524)
0808 00:39:58.790804 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.790864 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:39:58.791280 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.791193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59532)
0808 00:39:58.791233 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.791260 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:39:58.791394 24717 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38539: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
W20260808 00:39:58.791998 24874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:39:58.792042 24874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:39:58.792054 24874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:39:58.792068 24874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:39:58.792091 24874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:39:58.793612 24874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:36971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:42841,127.1.208.61:37653,127.1.208.60:38539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:38539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:39:58.793838 24874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:39:58.794047 24874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:39:58.798621 24887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:58.798621 24885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:39:58.798651 24883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:39:58.798810 24874 server_base.cc:1034] running on GCE node
I20260808 00:39:58.799118 24874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:39:58.799388 24874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:39:58.800529 24874 hybrid_clock.cc:648] HybridClock initialized: now 1786149598800514 us; error 33 us; skew 500 ppm
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:39:58.802968 24874 init.cc:377] Logged in from keytab as kudu/127.1.208.60@KRBTEST.COM (short username kudu)
I20260808 00:39:58.803351 24874 webserver.cc:460] Webserver started at http://127.1.208.60:45845/ using document root <none> and password file <none>
I20260808 00:39:58.803517 24874 fs_manager.cc:359] Metadata directory not provided
I20260808 00:39:58.803556 24874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:39:58.803741 24874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:39:58.804750 24874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data/instance:
uuid: "6e84a8eb0b504053bafb12390bf29ec2"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "7c3571ffa3dc65341b09a28ef55983e9"
server_key_iv: "1a15ef31a323e02b300973f64c1cd123"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.805045 24874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal/instance:
uuid: "6e84a8eb0b504053bafb12390bf29ec2"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "7c3571ffa3dc65341b09a28ef55983e9"
server_key_iv: "1a15ef31a323e02b300973f64c1cd123"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.806275 24874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:39:58.807004 24897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:39:58.807173 24874 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:39:58.807260 24874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "6e84a8eb0b504053bafb12390bf29ec2"
format_stamp: "Formatted at 2026-08-08 00:39:58 on dist-test-slave-gjw7"
server_key: "7c3571ffa3dc65341b09a28ef55983e9"
server_key_iv: "1a15ef31a323e02b300973f64c1cd123"
server_key_version: "encryptionkey@0"
I20260808 00:39:58.807333 24874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:39:58.812633 24864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } attempt: 2
I20260808 00:39:58.813009 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.812897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59544)
0808 00:39:58.812948 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.812979 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:39:58.813370 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.813259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59554)
0808 00:39:58.813324 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.813350 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 00:39:58.813447 24864 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38539: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38539: connect: Connection refused (error 111)
I20260808 00:39:58.819617 24874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:39:58.821245 24874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:39:58.821424 24874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:39:58.825274 24874 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.60:38539
I20260808 00:39:58.825348 24988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.60:38539 every 8 connection(s)
I20260808 00:39:58.825656 24874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
I20260808 00:39:58.826484 24990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:39:58.827993 24990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:39:58.833184 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 24874
I20260808 00:39:58.833309 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786149440113891-1856-0/minicluster-data/master-2/wal/instance
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:39:58.833498 1856 external_mini_cluster.cc:1428] Setting key 561f5bd589f64f1e312388a4df73a9c3
I20260808 00:39:58.836503 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.828696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.1.208.62:42841)
0808 00:39:58.828759 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.828762 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.828954 (+ 192us) server_negotiation.cc:407] Connection header received
0808 00:39:58.829461 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.829462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.829474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.829527 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.830125 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.830626 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.831591 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.831794 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.831851 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.833982 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.833983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.833984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.833992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.835252 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.835667 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.835668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.835668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.835743 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.835952 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.835952 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.835953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.836078 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.836139 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.836373 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.836438 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:39:58.836719 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.828644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:42841 (local address 127.0.0.1:42086)
0808 00:39:58.828839 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.828874 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.829368 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.829595 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.829599 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.829848 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.830038 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.830042 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.830709 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.830711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.831510 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.831512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.831868 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.831870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.831980 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.832420 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.832428 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.833860 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.835333 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.835339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.835341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.835597 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.835804 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.835805 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.835805 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.835894 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.836130 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.836133 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.836237 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.836357 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.836503 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":99,"threads_started":1}
I20260808 00:39:58.837225 24990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:39:58.841961 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.833856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59562 (local address 127.1.208.60:38539)
0808 00:39:58.834023 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.834027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.834047 (+ 20us) server_negotiation.cc:407] Connection header received
0808 00:39:58.834087 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.834090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.834102 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.834166 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.834714 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.835410 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.836287 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.836684 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.836744 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.839243 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.839251 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.839252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.839258 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.840657 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.841176 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.841179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.841180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.841233 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.841427 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.841428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.841429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.841564 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.841687 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.841783 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.841862 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260808 00:39:58.843355 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.837457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:37653 (local address 127.0.0.1:45936)
0808 00:39:58.837500 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.837509 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.837546 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.837679 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.837679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.837815 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.837872 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.837873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.838436 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.838437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.838986 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.838989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.839366 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.839367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.839439 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.839757 (+ 318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.839762 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.840889 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.842412 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.842413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.842413 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.842607 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.842809 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.842810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.842810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.842873 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.843087 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.843087 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.843128 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.843186 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.843269 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:39:58.843380 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.837522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45936 (local address 127.1.208.61:37653)
0808 00:39:58.837563 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.837565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.837589 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:39:58.837613 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.837614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.837621 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.837656 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.837933 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.838368 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.839073 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.839296 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.839352 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.840960 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.840962 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.840962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.840970 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.842344 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.842692 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.842693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.842694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.842751 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.842927 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.842928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.842928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.843034 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.843087 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.843216 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.843286 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:39:58.843699 24990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:39:58.849618 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.842666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42096 (local address 127.1.208.62:42841)
0808 00:39:58.842726 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.842728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.842735 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:39:58.842754 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.842754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.842763 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.842807 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.843196 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.843763 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.844576 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.844854 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.844915 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.847061 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.847062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.847062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.847069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.848550 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.848851 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.848852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.848852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.848911 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.849083 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.849084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.849084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.849192 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.849419 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.849458 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.849530 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:39:58.849874 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.843913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59564)
0808 00:39:58.843949 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.843957 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.843987 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.844081 (+ 94us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.844081 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.844235 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.844266 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.844267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.845057 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.845058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.845792 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.845795 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.846119 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.846120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.846175 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.846492 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.846496 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.847571 (+ 1075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.848907 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.848908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.848908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.849094 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.849253 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.849253 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.849254 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.849350 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.849597 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.849598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.849642 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.849704 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.849785 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260808 00:39:58.850111 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.843964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59564 (local address 127.1.208.60:38539)
0808 00:39:58.843997 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.843999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.844005 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:39:58.844025 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.844026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.844035 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.844074 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.844335 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.844989 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.845872 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.846056 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.846099 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.847647 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.847648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.847649 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.847656 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.848836 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.849156 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.849157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.849157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.849204 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.849413 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.849414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.849414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.849534 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.849601 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.849967 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.850036 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:39:58.851573 24990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: Bootstrap starting.
I20260808 00:39:58.852684 24990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:58.853034 24990 log.cc:824] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:58.853942 24990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: No bootstrap required, opened a new log
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:39:58.855453 24990 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.855581 24990 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:58.855613 24990 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e84a8eb0b504053bafb12390bf29ec2, State: Initialized, Role: FOLLOWER
I20260808 00:39:58.855743 24990 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.856029 24999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:58.856082 24990 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:58.856123 24999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:39:58.857275 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.850327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45942 (local address 127.1.208.61:37653)
0808 00:39:58.850363 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.850364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.850372 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:58.850393 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.850394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.850403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.850459 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.850972 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.851431 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.852279 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.852452 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.852501 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.854769 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.854771 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.854771 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.854779 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.856116 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.856442 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.856444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.856444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.856512 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.856707 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.856708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.856709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.856861 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.857053 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.857094 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.857180 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:39:58.857995 25018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:39:58.858052 25018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:39:58.858853 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:39:58.858625 24874 master_runner.cc:186] Error getting master registration for 127.1.208.62:42841: OK, instance_id { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" instance_seqno: 1786149598525246 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:39:58.861088 24874 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:39:58.866530 24717 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } attempt: 2
I20260808 00:39:58.867658 24997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.861244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.1.208.60:38539)
0808 00:39:58.861328 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.861330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.861338 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:58.861359 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.861359 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.861368 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.861428 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.862072 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.862759 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.863493 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.863750 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.863814 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.865001 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.865002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.865002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.865009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.866400 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.866771 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.866772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.866773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.866834 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.867039 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.867041 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.867041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.867214 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.867293 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.867498 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.867572 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:39:58.868829 24870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.860666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.1.208.61:37653)
0808 00:39:58.860705 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.860706 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.860714 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:58.860734 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.860734 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.860743 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.860802 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.861559 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.862249 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.863179 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.863475 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.863553 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.864959 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.864960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.864960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.864967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.866679 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.867341 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.867343 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.867343 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.867402 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.867600 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.867600 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.867601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.867720 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.868419 (+ 699us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.868650 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.868733 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:39:58.874787 24721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.866866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59584)
0808 00:39:58.866915 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.866924 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.866957 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.867289 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.867290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.867455 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.867506 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.867509 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.868846 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.868847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.869584 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.869585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.870125 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.870127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.870184 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.870560 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.870564 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.872241 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.873783 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.873784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.873785 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.873966 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.874171 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.874172 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.874172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.874260 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.874511 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.874512 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.874554 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.874599 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.874679 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260808 00:39:58.875010 24722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.861512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42104 (local address 127.1.208.62:42841)
0808 00:39:58.861554 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.861556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.868174 (+ 6618us) server_negotiation.cc:407] Connection header received
0808 00:39:58.868278 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.868279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.868290 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.868318 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.868821 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.869578 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.870447 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.870749 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.870815 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.872007 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.872008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.872009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.872015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.873612 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.873952 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.873953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.873953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.874005 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.874172 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.874172 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.874172 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.874300 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.874811 (+ 511us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.874850 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.874922 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:39:58.875221 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.866977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.1.208.60:38539)
0808 00:39:58.867138 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.867140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.867146 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:39:58.867163 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.867163 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.867183 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.867573 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.867600 (+ 27us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.868767 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.869749 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.870047 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.872051 (+ 2004us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.872306 (+ 255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.872307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.872307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.872315 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.873676 (+ 1361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.874036 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.874037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.874037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.874097 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.874318 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.874319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.874319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.874434 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.874957 (+ 523us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.875010 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.875098 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":94,"threads_started":1}
I20260808 00:39:58.876858 24717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: Bootstrap starting.
I20260808 00:39:58.877637 24717 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:58.877977 24717 log.cc:824] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:58.878785 24717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: No bootstrap required, opened a new log
I20260808 00:39:58.881150 24717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.881318 24717 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:58.881376 24717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ad71b48ee5f4186842dfa407b844cc5, State: Initialized, Role: FOLLOWER
I20260808 00:39:58.881498 24717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.881768 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:58.881824 24717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:58.881872 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:39:58.883723 25053 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:39:58.883829 25053 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:39:58.884342 24609 master_runner.cc:186] Error getting master registration for 127.1.208.61:37653: OK, instance_id { permanent_uuid: "7123513f02ed4591bc791322c8804103" instance_seqno: 1786149598673197 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:39:58.887279 24609 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:39:58.904594 24864 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } attempt: 3
Aug 08 00:39:58 dist-test-slave-gjw7 krb5kdc[24592](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149598, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:39:58.913237 24867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.904975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38539 (local address 127.0.0.1:59590)
0808 00:39:58.905031 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:39:58.905043 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:39:58.905082 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:39:58.905488 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:39:58.905489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:39:58.905693 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:39:58.905751 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.905753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.906722 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.906724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.907546 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:39:58.907549 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.908051 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.908052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:39:58.908116 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.908623 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:39:58.908628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:39:58.910245 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:39:58.912156 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.912157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.912158 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.912380 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.912583 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:39:58.912584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:39:58.912584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:39:58.912649 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:39:58.912904 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:39:58.912905 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:39:58.912956 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 00:39:58.913026 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 00:39:58.913127 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 00:39:58.913239 25033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:39:58.905017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.1.208.60:38539)
0808 00:39:58.905321 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 00:39:58.905333 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:39:58.905341 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:39:58.905362 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:39:58.905363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:39:58.905374 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:39:58.905457 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:39:58.905835 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.906634 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.907633 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:39:58.907972 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:39:58.908045 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:39:58.910335 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:39:58.910336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:39:58.910337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:39:58.910345 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:39:58.912069 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.912463 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.912464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.912465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.912523 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:39:58.912703 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:39:58.912703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:39:58.912704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:39:58.912844 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:39:58.912909 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:39:58.913054 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:39:58.913140 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260808 00:39:58.915445 24864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: Bootstrap starting.
I20260808 00:39:58.916224 24864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: Neither blocks nor log segments found. Creating new log.
I20260808 00:39:58.916548 24864 log.cc:824] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: Log is configured to *not* fsync() on all Append() calls
I20260808 00:39:58.917657 24864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: No bootstrap required, opened a new log
I20260808 00:39:58.919703 24864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.919845 24864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:39:58.919878 24864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7123513f02ed4591bc791322c8804103, State: Initialized, Role: FOLLOWER
I20260808 00:39:58.920049 24864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:58.920379 24864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:39:58.920343 25060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:58.920547 25060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:39:58.920989 24777 tablet.cc:2369] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:39:58.922356 25089 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:39:58.922477 25089 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:39:58.924384 24728 master_runner.cc:263] No leader master found from master 7123513f02ed4591bc791322c8804103
I20260808 00:39:58.924443 24728 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 00:39:59.023135 24639 tablet.cc:2369] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:39:59.072007 24908 tablet.cc:2369] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:39:59.092757 25060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:39:59.092900 25060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:59.093173 25060 leader_election.cc:288] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ad71b48ee5f4186842dfa407b844cc5 (127.1.208.62:42841), 6e84a8eb0b504053bafb12390bf29ec2 (127.1.208.60:38539)
I20260808 00:39:59.093493 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7123513f02ed4591bc791322c8804103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e84a8eb0b504053bafb12390bf29ec2" is_pre_election: true
I20260808 00:39:59.093487 24681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7123513f02ed4591bc791322c8804103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ad71b48ee5f4186842dfa407b844cc5" is_pre_election: true
I20260808 00:39:59.093636 24681 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7123513f02ed4591bc791322c8804103 in term 0.
I20260808 00:39:59.093650 24940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7123513f02ed4591bc791322c8804103 in term 0.
I20260808 00:39:59.093866 24755 leader_election.cc:302] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7123513f02ed4591bc791322c8804103, 8ad71b48ee5f4186842dfa407b844cc5; no voters:
I20260808 00:39:59.094064 25060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:39:59.094126 25060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:39:59.094142 25060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:59.094834 25060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:59.094964 25060 leader_election.cc:288] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [CANDIDATE]: Term 1 election: Requested vote from peers 8ad71b48ee5f4186842dfa407b844cc5 (127.1.208.62:42841), 6e84a8eb0b504053bafb12390bf29ec2 (127.1.208.60:38539)
I20260808 00:39:59.095132 24940 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7123513f02ed4591bc791322c8804103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e84a8eb0b504053bafb12390bf29ec2"
I20260808 00:39:59.095139 24681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7123513f02ed4591bc791322c8804103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ad71b48ee5f4186842dfa407b844cc5"
I20260808 00:39:59.095214 24681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:59.095214 24940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:39:59.095952 24681 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7123513f02ed4591bc791322c8804103 in term 1.
I20260808 00:39:59.095981 24940 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7123513f02ed4591bc791322c8804103 in term 1.
I20260808 00:39:59.096124 24755 leader_election.cc:302] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7123513f02ed4591bc791322c8804103, 8ad71b48ee5f4186842dfa407b844cc5; no voters:
I20260808 00:39:59.096230 25060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:39:59.096407 25060 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [term 1 LEADER]: Becoming Leader. State: Replica: 7123513f02ed4591bc791322c8804103, State: Running, Role: LEADER
I20260808 00:39:59.096500 25060 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } }
I20260808 00:39:59.096905 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7123513f02ed4591bc791322c8804103. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.096977 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:59.097146 25093 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:39:59.097345 25093 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:39:59.098625 24681 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [term 1 FOLLOWER]: Refusing update from remote peer 7123513f02ed4591bc791322c8804103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:59.098868 24940 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [term 1 FOLLOWER]: Refusing update from remote peer 7123513f02ed4591bc791322c8804103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:39:59.098929 25060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:39:59.099109 25060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:39:59.100154 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7123513f02ed4591bc791322c8804103. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.100252 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:39:59.100214 24999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7123513f02ed4591bc791322c8804103. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.100299 24999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:39:59.101128 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.101204 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:59.101243 25091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.101274 25091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103 [sys.catalog]: This master's current role is: LEADER
I20260808 00:39:59.101311 25095 mvcc.cc:205] Tried to move back new op lower bound from 7316068757905821696 to 7316068757899763712. Current Snapshot: MvccSnapshot[applied={T|T < 7316068757905821696}]
I20260808 00:39:59.101334 25094 mvcc.cc:205] Tried to move back new op lower bound from 7316068757905821696 to 7316068757899763712. Current Snapshot: MvccSnapshot[applied={T|T < 7316068757905821696}]
I20260808 00:39:59.101274 24999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.101380 24999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:39:59.101392 25038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7123513f02ed4591bc791322c8804103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ad71b48ee5f4186842dfa407b844cc5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 42841 } } peers { permanent_uuid: "7123513f02ed4591bc791322c8804103" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 37653 } } peers { permanent_uuid: "6e84a8eb0b504053bafb12390bf29ec2" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38539 } } }
I20260808 00:39:59.101449 25038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:39:59.101526 25093 catalog_manager.cc:1347] Generated new cluster ID: 1b950467796b497f8875e059c91c0483
I20260808 00:39:59.101567 25093 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:39:59.101562 25096 mvcc.cc:205] Tried to move back new op lower bound from 7316068757905821696 to 7316068757899763712. Current Snapshot: MvccSnapshot[applied={T|T < 7316068757905821696}]
I20260808 00:39:59.130390 25093 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:39:59.131094 25093 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:39:59.138118 25093 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7123513f02ed4591bc791322c8804103: Generated new TSK 0
I20260808 00:39:59.138330 25093 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:39:59.858505 25018 catalog_manager.cc:1259] Loaded cluster ID: 1b950467796b497f8875e059c91c0483
I20260808 00:39:59.858593 25018 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: loading cluster ID for follower catalog manager: success
I20260808 00:39:59.859506 25018 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: acquiring CA information for follower catalog manager: success
I20260808 00:39:59.859819 25018 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6e84a8eb0b504053bafb12390bf29ec2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:39:59.884398 25053 catalog_manager.cc:1259] Loaded cluster ID: 1b950467796b497f8875e059c91c0483
I20260808 00:39:59.884461 25053 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: loading cluster ID for follower catalog manager: success
I20260808 00:39:59.885695 25053 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: acquiring CA information for follower catalog manager: success
I20260808 00:39:59.886092 25053 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8ad71b48ee5f4186842dfa407b844cc5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:40:00.194635 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 24609
I20260808 00:40:00.211414 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 24728
I20260808 00:40:00.222271 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 24874
2026-08-08T00:40:00Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3133 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:40:00 dist-test-slave-gjw7 krb5kdc[25117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:40:00 dist-test-slave-gjw7 krb5kdc[25117](info): set up 2 sockets
Aug 08 00:40:00 dist-test-slave-gjw7 krb5kdc[25117](info): commencing operation
krb5kdc: starting...
W20260808 00:40:01.408197 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:40:01.523979 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43387
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:43387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:01.604872 25140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:01.605079 25140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:01.605100 25140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:01.606209 25140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:01.606243 25140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:01.606256 25140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:01.606267 25140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:01.606285 25140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:01.607699 25140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:43387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:01.607882 25140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:01.608072 25140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:01.612643 25152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:01.612641 25146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:01.612725 25148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:01.612812 25140 server_base.cc:1034] running on GCE node
I20260808 00:40:01.613178 25140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:01.613490 25140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:01.614625 25140 hybrid_clock.cc:648] HybridClock initialized: now 1786149601614614 us; error 33 us; skew 500 ppm
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149601, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:01.616938 25140 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:01.617379 25140 webserver.cc:460] Webserver started at http://127.1.208.62:33755/ using document root <none> and password file <none>
I20260808 00:40:01.617586 25140 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:01.617625 25140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:01.617720 25140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:01.618700 25140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "ed5106c5943b41eca02d4e2298db49ba"
format_stamp: "Formatted at 2026-08-08 00:40:01 on dist-test-slave-gjw7"
server_key: "13e46b5d6bda2052f6c39007268b8618"
server_key_iv: "57ccdf00fd550156326c76d44ec6a4f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:01.618996 25140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "ed5106c5943b41eca02d4e2298db49ba"
format_stamp: "Formatted at 2026-08-08 00:40:01 on dist-test-slave-gjw7"
server_key: "13e46b5d6bda2052f6c39007268b8618"
server_key_iv: "57ccdf00fd550156326c76d44ec6a4f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:01.620170 25140 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:40:01.620846 25163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:01.620993 25140 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:01.621079 25140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "ed5106c5943b41eca02d4e2298db49ba"
format_stamp: "Formatted at 2026-08-08 00:40:01 on dist-test-slave-gjw7"
server_key: "13e46b5d6bda2052f6c39007268b8618"
server_key_iv: "57ccdf00fd550156326c76d44ec6a4f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:01.621138 25140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:01.644865 25140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:01.646701 25140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:01.646842 25140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:01.650830 25140 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:43387
I20260808 00:40:01.650884 25265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:43387 every 8 connection(s)
I20260808 00:40:01.651211 25140 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:01.652055 25266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:01.654359 25266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba: Bootstrap starting.
I20260808 00:40:01.654884 25266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:01.655130 25266 log.cc:824] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:01.655791 25266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba: No bootstrap required, opened a new log
I20260808 00:40:01.657393 25266 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } }
I20260808 00:40:01.657533 25266 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:01.657554 25266 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed5106c5943b41eca02d4e2298db49ba, State: Initialized, Role: FOLLOWER
I20260808 00:40:01.657658 25266 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } }
I20260808 00:40:01.657706 25266 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:01.657730 25266 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:01.657760 25266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:01.658224 25266 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } }
I20260808 00:40:01.658342 25266 leader_election.cc:302] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed5106c5943b41eca02d4e2298db49ba; no voters:
I20260808 00:40:01.658434 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 25140
I20260808 00:40:01.658511 25266 leader_election.cc:288] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:40:01.658532 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:40:01.658560 25274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:01.658725 1856 external_mini_cluster.cc:1428] Setting key 39ce417741f00a78dce9ba2d0ca1ac32
I20260808 00:40:01.658725 25274 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [term 1 LEADER]: Becoming Leader. State: Replica: ed5106c5943b41eca02d4e2298db49ba, State: Running, Role: LEADER
I20260808 00:40:01.658763 25266 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:01.658810 25274 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } }
I20260808 00:40:01.659441 25275 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed5106c5943b41eca02d4e2298db49ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } } }
I20260808 00:40:01.659528 25275 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:01.659736 25289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:40:01.659864 25276 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed5106c5943b41eca02d4e2298db49ba. Latest consensus state: current_term: 1 leader_uuid: "ed5106c5943b41eca02d4e2298db49ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed5106c5943b41eca02d4e2298db49ba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43387 } } }
I20260808 00:40:01.659992 25276 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:01.660302 25289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:01.661871 25289 catalog_manager.cc:1347] Generated new cluster ID: 48cd13032703427599f833e9ef4c4706
I20260808 00:40:01.661926 25289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:01.665863 25289 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:40:01.666497 25289 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:40:01.670545 25283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:01.659161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.1.208.62:43387)
0808 00:40:01.659345 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:01.659355 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:01.659391 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:40:01.659520 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:01.659527 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:01.659583 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:01.660555 (+ 972us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:01.660704 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:01.661924 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:01.662973 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:01.663565 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:01.663656 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:01.666394 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:01.666398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:01.666398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:01.666418 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:01.668936 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:01.669345 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:01.669348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:01.669348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:01.669472 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:01.669653 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:01.669653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:01.669654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:01.669930 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:01.670105 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:01.670174 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 00:40:01.670323 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260808 00:40:01.671166 25289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed5106c5943b41eca02d4e2298db49ba: Generated new TSK 0
I20260808 00:40:01.671320 25289 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:40:01.671509 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:40:01.674226 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 25140
2026-08-08T00:40:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1456 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25311](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25311](info): set up 2 sockets
Aug 08 00:40:01 dist-test-slave-gjw7 krb5kdc[25311](info): commencing operation
krb5kdc: starting...
W20260808 00:40:02.854851 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:40:02 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:40:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:40:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:40:02.972630 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:03.054703 25331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.054904 25331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.055027 25331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.056327 25331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:03.056370 25331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.056383 25331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.056394 25331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:03.056408 25331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:03.057926 25331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.058130 25331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.058316 25331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:03.062865 25351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.062862 25344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.062875 25345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.063135 25331 server_base.cc:1034] running on GCE node
I20260808 00:40:03.063361 25331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.063616 25331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.064769 25331 hybrid_clock.cc:648] HybridClock initialized: now 1786149603064747 us; error 33 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.067094 25331 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:03.067502 25331 webserver.cc:460] Webserver started at http://127.1.208.62:41865/ using document root <none> and password file <none>
I20260808 00:40:03.067695 25331 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.067741 25331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.067874 25331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:03.068879 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "8ce6d00b2d964591a9cc80920ac77d82"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "f7f5064bfbfe1f0bc82857a9aabd4f41"
server_key_iv: "928a0ca7482173d88d7532c1287606f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.069209 25331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "8ce6d00b2d964591a9cc80920ac77d82"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "f7f5064bfbfe1f0bc82857a9aabd4f41"
server_key_iv: "928a0ca7482173d88d7532c1287606f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.070461 25331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:40:03.071110 25359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.071259 25331 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:40:03.071344 25331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "8ce6d00b2d964591a9cc80920ac77d82"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "f7f5064bfbfe1f0bc82857a9aabd4f41"
server_key_iv: "928a0ca7482173d88d7532c1287606f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.071410 25331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.087602 25331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.089530 25331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.089669 25331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.093570 25331 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38045
I20260808 00:40:03.093611 25451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38045 every 8 connection(s)
I20260808 00:40:03.093957 25331 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:03.094920 25452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:03.097134 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 25331
I20260808 00:40:03.097229 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:40:03.097275 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap starting.
I20260808 00:40:03.097455 1856 external_mini_cluster.cc:1428] Setting key dddf2c61d1d43521e2027d838097656b
I20260808 00:40:03.097988 25452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:03.098308 25452 log.cc:824] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.099035 25452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: No bootstrap required, opened a new log
I20260808 00:40:03.101176 25452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.101368 25452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:03.101409 25452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Initialized, Role: FOLLOWER
I20260808 00:40:03.101536 25452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.101634 25452 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:03.101665 25452 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:03.101712 25452 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:03.102263 25452 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.102380 25452 leader_election.cc:302] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ce6d00b2d964591a9cc80920ac77d82; no voters:
I20260808 00:40:03.102536 25452 leader_election.cc:288] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:40:03.102571 25463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:03.102782 25463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 LEADER]: Becoming Leader. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Running, Role: LEADER
I20260808 00:40:03.102797 25452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:03.102949 25463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.103693 25472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ce6d00b2d964591a9cc80920ac77d82. Latest consensus state: current_term: 1 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:03.103772 25472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:03.104066 25489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:03.104439 25467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:03.104517 25467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:03.104662 25489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:03.106226 25489 catalog_manager.cc:1347] Generated new cluster ID: 45c8f3f06f1f4fdb9f6478071b9a974d
I20260808 00:40:03.106280 25489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:40:03.108732 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.097944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49240 (local address 127.1.208.62:38045)
0808 00:40:03.098148 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.098156 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.098180 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:40:03.098701 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.098711 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.098774 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.098896 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:03.099686 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.100536 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.101595 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.102079 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.102156 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.104912 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:03.104916 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:03.104916 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:03.104927 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:03.107130 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.107523 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.107526 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.107527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.107657 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.107841 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.107842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.107843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.108151 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:03.108307 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.108379 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.108517 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
I20260808 00:40:03.116287 25489 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:40:03.116784 25489 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:40:03.121959 25489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Generated new TSK 0
I20260808 00:40:03.122116 25489 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:40:03.150206 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:03.232738 25512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.232951 25512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.232985 25512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.234166 25512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.234210 25512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.234246 25512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:40:03.235724 25512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.235983 25512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.236200 25512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:03.240950 25522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.241034 25512 server_base.cc:1034] running on GCE node
W20260808 00:40:03.240969 25526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.240934 25520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.241384 25512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.241595 25512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.242725 25512 hybrid_clock.cc:648] HybridClock initialized: now 1786149603242713 us; error 29 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.245133 25512 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:40:03.245571 25512 webserver.cc:460] Webserver started at http://127.1.208.1:37105/ using document root <none> and password file <none>
I20260808 00:40:03.245785 25512 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.245839 25512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.245967 25512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:03.246989 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "ae824ee7574546c1823995d6be3596ce"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "b690d367b84b30d2ee73ee318b1353d8"
server_key_iv: "a396a00438bbf9842ced7467aa972275"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.247319 25512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "ae824ee7574546c1823995d6be3596ce"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "b690d367b84b30d2ee73ee318b1353d8"
server_key_iv: "a396a00438bbf9842ced7467aa972275"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.248530 25512 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:40:03.249159 25541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.249342 25512 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:03.249436 25512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ae824ee7574546c1823995d6be3596ce"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "b690d367b84b30d2ee73ee318b1353d8"
server_key_iv: "a396a00438bbf9842ced7467aa972275"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.249491 25512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.266251 25512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.268262 25512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.268422 25512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.268635 25512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:03.268941 25512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:03.268975 25512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.269006 25512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:03.269022 25512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.274600 25512 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:44009
I20260808 00:40:03.274660 25738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:44009 every 8 connection(s)
I20260808 00:40:03.274959 25512 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:40:03.275671 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 25512
I20260808 00:40:03.275764 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:40:03.275986 1856 external_mini_cluster.cc:1428] Setting key 9cbaf94d92611af8c459c41ba13979f2
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:03.284299 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.275690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60239 (local address 127.1.208.62:38045)
0808 00:40:03.275760 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.275762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.275966 (+ 204us) server_negotiation.cc:407] Connection header received
0808 00:40:03.277061 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.277062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.277076 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.277125 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:03.277821 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.278401 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.279393 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.279637 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.279697 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.281771 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:03.281772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:03.281772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:03.281780 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:03.283016 (+ 1236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.283403 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.283405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.283405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.283460 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.283684 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.283685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.283685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.283791 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:03.283846 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.284110 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.284205 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
I20260808 00:40:03.284457 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.275645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:60239)
0808 00:40:03.275820 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.275863 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.276957 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.277200 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.277204 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.277528 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:03.277723 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.277729 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.278497 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.278498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.279303 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.279305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.279731 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.279733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.279859 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.280320 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:03.280335 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:03.281680 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:03.283093 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.283100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.283103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.283337 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.283516 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.283517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.283517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.283629 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.283850 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:03.283856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:03.283992 (+ 136us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.284080 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.284240 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260808 00:40:03.284934 25739 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:03.285032 25739 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:03.285207 25739 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:03.285802 25396 ts_manager.cc:194] Registered new tserver with Master: ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009)
I20260808 00:40:03.286422 25396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:60239
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:40:03.303567 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:03.390022 25748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.390236 25748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.390259 25748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.391429 25748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.391474 25748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.391501 25748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:40:03.392959 25748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.393179 25748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.393435 25748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:03.398262 25756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.398401 25748 server_base.cc:1034] running on GCE node
W20260808 00:40:03.398442 25759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.398268 25754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.398681 25748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.398955 25748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.400089 25748 hybrid_clock.cc:648] HybridClock initialized: now 1786149603400076 us; error 31 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.402519 25748 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:40:03.402940 25748 webserver.cc:460] Webserver started at http://127.1.208.2:37361/ using document root <none> and password file <none>
I20260808 00:40:03.403151 25748 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.403199 25748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.403326 25748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:03.404395 25748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "571f38c1f2054a5ea94836b0e100e523"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "2e4838c5067ec05c23a7b4145a93e7c3"
server_key_iv: "85192ccd48087d5bd15a947f9cfe8ead"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.404754 25748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "571f38c1f2054a5ea94836b0e100e523"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "2e4838c5067ec05c23a7b4145a93e7c3"
server_key_iv: "85192ccd48087d5bd15a947f9cfe8ead"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.406075 25748 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:40:03.406736 25771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.406881 25748 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.406966 25748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "571f38c1f2054a5ea94836b0e100e523"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "2e4838c5067ec05c23a7b4145a93e7c3"
server_key_iv: "85192ccd48087d5bd15a947f9cfe8ead"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.407027 25748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.415203 25748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.416960 25748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.417090 25748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.417330 25748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:03.417680 25748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:03.417717 25748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.417747 25748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:03.417764 25748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.423461 25748 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:34559
I20260808 00:40:03.423523 25967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:34559 every 8 connection(s)
I20260808 00:40:03.423815 25748 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:40:03.429360 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 25748
I20260808 00:40:03.429462 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:40:03.429663 1856 external_mini_cluster.cc:1428] Setting key 046212ef2c54ea76098d9e3e70b9cde9
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:03.432901 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.424514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:43251 (local address 127.1.208.62:38045)
0808 00:40:03.424571 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.424573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.424750 (+ 177us) server_negotiation.cc:407] Connection header received
0808 00:40:03.425279 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.425281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.425308 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.425384 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:03.426326 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.426824 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.427792 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.427992 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.428048 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.430340 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:03.430341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:03.430342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:03.430349 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:03.431550 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.431922 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.431923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.431924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.431985 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.432299 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.432300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.432301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.432439 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:03.432504 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.432723 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.432808 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:40:03.432996 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.424464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:43251)
0808 00:40:03.424633 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.424669 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.425173 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.425437 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.425441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.426032 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:03.426224 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.426229 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.426916 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.426917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.427703 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.427706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.428062 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.428063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.428179 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.428648 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:03.428656 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:03.430245 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:03.431625 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.431632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.431634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.431848 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.432075 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.432076 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.432076 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.432233 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.432496 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:03.432500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:03.432613 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.432691 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.432822 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260808 00:40:03.433524 25970 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:03.433622 25970 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:03.433804 25970 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:03.434194 25396 ts_manager.cc:194] Registered new tserver with Master: 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
I20260808 00:40:03.434659 25396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:43251
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:40:03.455946 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:03.537336 25983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.537540 25983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.537571 25983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.538671 25983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.538712 25983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.538745 25983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:40:03.540160 25983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.540395 25983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.540606 25983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:03.545589 25998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.545583 25993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.545583 25994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.545818 25983 server_base.cc:1034] running on GCE node
I20260808 00:40:03.546000 25983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.546221 25983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.547358 25983 hybrid_clock.cc:648] HybridClock initialized: now 1786149603547339 us; error 30 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.552470 25983 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:40:03.552901 25983 webserver.cc:460] Webserver started at http://127.1.208.3:35489/ using document root <none> and password file <none>
I20260808 00:40:03.553126 25983 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.553175 25983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.553331 25983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:03.554368 25983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "eb0406fafad5fc932797cb69834266c8"
server_key_iv: "e3b0a630f119ed8b1a3ccbb8901c108c"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.554723 25983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "eb0406fafad5fc932797cb69834266c8"
server_key_iv: "e3b0a630f119ed8b1a3ccbb8901c108c"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.555991 25983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:40:03.556646 26009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.556797 25983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:40:03.556882 25983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "eb0406fafad5fc932797cb69834266c8"
server_key_iv: "e3b0a630f119ed8b1a3ccbb8901c108c"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.556950 25983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.571139 25983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.573156 25983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.573361 25983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.573653 25983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:03.573984 25983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:03.574018 25983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.574041 25983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:03.574081 25983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.579859 25983 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:38449
I20260808 00:40:03.579945 26200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:38449 every 8 connection(s)
I20260808 00:40:03.580206 25983 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:40:03.580914 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 25983
I20260808 00:40:03.581000 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:40:03.581219 1856 external_mini_cluster.cc:1428] Setting key c12e2cd0d0ffd6b90dbde143a9684ce2
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:03.588572 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.580890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:56731 (local address 127.1.208.62:38045)
0808 00:40:03.580949 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.580951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.581182 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:40:03.581787 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.581787 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.581799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.581834 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:03.582845 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.583344 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.584280 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.584470 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.584526 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.586468 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:03.586469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:03.586470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:03.586477 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:03.587547 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.587878 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.587878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.587879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.587927 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.588100 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.588101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.588101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.588194 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:03.588238 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.588449 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.588506 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:40:03.588735 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.580835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:56731)
0808 00:40:03.581031 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.581081 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.581697 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.581910 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.581915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.582546 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:03.582743 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.582749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.583412 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.583413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.584200 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.584202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.584531 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.584532 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.584629 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.585036 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:03.585044 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:03.586391 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:03.587610 (+ 1219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.587616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.587617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.587823 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.587970 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:03.587970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:03.587970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:03.588060 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:03.588234 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:03.588237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:03.588351 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.588425 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.588537 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":91,"threads_started":1}
I20260808 00:40:03.589151 26203 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:03.589228 26203 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:03.589401 26203 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:03.589736 25396 ts_manager.cc:194] Registered new tserver with Master: 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449)
I20260808 00:40:03.590119 25396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:56731
I20260808 00:40:03.592844 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:40:03.599584 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.594172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49246 (local address 127.1.208.62:38045)
0808 00:40:03.594219 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.594221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.594316 (+ 95us) server_negotiation.cc:407] Connection header received
0808 00:40:03.594380 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.594381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.594390 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.594439 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:03.594988 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.595448 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.596159 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.596340 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.596392 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.597528 (+ 1136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:03.597529 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:03.597529 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:03.597538 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:03.598569 (+ 1031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.598867 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.598869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.598869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.598920 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:03.599079 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:03.599080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:03.599080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:03.599218 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:03.599306 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.599424 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.599498 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:40:03.601027 25396 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:40:03.602671 25396 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:40:03.608662 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.605067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45232 (local address 127.1.208.1:44009)
0808 00:40:03.605219 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.605224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.605234 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:40:03.605277 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.605279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.605328 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.605403 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:03.605550 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606359 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.607638 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.608289 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.608349 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.608408 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.608547 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.608587 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260808 00:40:03.608839 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.604794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44009 (local address 127.0.0.1:45232)
0808 00:40:03.605091 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.605102 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.605155 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.605411 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.605414 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.605416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:03.605480 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.605482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606456 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.606458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.607543 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.607546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.608362 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.608362 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.608451 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.608463 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.608523 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.608574 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":2,"thread_start_us":54,"threads_started":1}
I20260808 00:40:03.608930 26219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.604794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:34559 (local address 127.0.0.1:45104)
0808 00:40:03.604959 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.604998 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.605063 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.605597 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.605598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.605599 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:03.605647 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.605648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606979 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.606980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.607819 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.607821 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.608700 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.608701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.608772 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.608776 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.608805 (+ 29us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.608843 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":55,"threads_started":1}
I20260808 00:40:03.609251 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.604916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.1.208.2:34559)
0808 00:40:03.605439 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.605443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.605452 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:40:03.605489 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.605491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.605506 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.605718 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:03.605800 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606870 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.607902 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.608599 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.609034 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.609099 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.609125 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.609168 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":89,"threads_started":1}
I20260808 00:40:03.609459 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.605159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56910 (local address 127.1.208.3:38449)
0808 00:40:03.605344 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.605348 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.605359 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:40:03.605397 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.605399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.605413 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.605488 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:03.605723 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606810 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.608067 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.608977 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.609035 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.609094 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.609327 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.609372 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":97,"threads_started":1}
I20260808 00:40:03.609583 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.604805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38449 (local address 127.0.0.1:56910)
0808 00:40:03.605238 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.605248 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.605312 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.605585 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.605587 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.605589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:03.605644 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.605645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.606930 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.606931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.607988 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.607990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.609159 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.609160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.609218 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.609222 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.609274 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.609520 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":51,"threads_started":1}
I20260808 00:40:03.609728 25633 tablet_service.cc:1467] Processing CreateTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed (DEFAULT_TABLE table=test-table [id=f003beb54b07458786b0ab195259d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:03.609933 25856 tablet_service.cc:1467] Processing CreateTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed (DEFAULT_TABLE table=test-table [id=f003beb54b07458786b0ab195259d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:03.610098 25633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768458a0c6d348ff8d1687f09b8d8eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:03.610250 25856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768458a0c6d348ff8d1687f09b8d8eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:03.612569 26232 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Bootstrap starting.
I20260808 00:40:03.613080 26233 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Bootstrap starting.
I20260808 00:40:03.613005 26088 tablet_service.cc:1467] Processing CreateTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed (DEFAULT_TABLE table=test-table [id=f003beb54b07458786b0ab195259d5de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:03.613242 26232 tablet_bootstrap.cc:654] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:03.613376 26088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 768458a0c6d348ff8d1687f09b8d8eed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:03.613744 26233 tablet_bootstrap.cc:654] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:03.613773 26232 log.cc:824] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.614286 26233 log.cc:824] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.614466 26232 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: No bootstrap required, opened a new log
I20260808 00:40:03.614529 26232 ts_tablet_manager.cc:1397] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:40:03.615872 26232 raft_consensus.cc:348] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.616060 26232 raft_consensus.cc:374] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:03.616110 26232 raft_consensus.cc:729] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae824ee7574546c1823995d6be3596ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:03.616189 26240 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Bootstrap starting.
I20260808 00:40:03.616259 26232 consensus_queue.cc:260] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.616600 25739 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
I20260808 00:40:03.616601 26232 ts_tablet_manager.cc:1428] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:40:03.616986 26240 tablet_bootstrap.cc:654] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:03.617130 26233 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: No bootstrap required, opened a new log
I20260808 00:40:03.617199 26233 ts_tablet_manager.cc:1397] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 00:40:03.617571 26240 log.cc:824] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.618300 26240 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: No bootstrap required, opened a new log
I20260808 00:40:03.618376 26240 ts_tablet_manager.cc:1397] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:40:03.619119 26233 raft_consensus.cc:348] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.619297 26233 raft_consensus.cc:374] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:03.619342 26233 raft_consensus.cc:729] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571f38c1f2054a5ea94836b0e100e523, State: Initialized, Role: FOLLOWER
I20260808 00:40:03.619503 26233 consensus_queue.cc:260] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.619784 26233 ts_tablet_manager.cc:1428] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 00:40:03.619808 25970 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
I20260808 00:40:03.620133 26240 raft_consensus.cc:348] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.620255 26240 raft_consensus.cc:374] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:03.620275 26240 raft_consensus.cc:729] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c6bb9ef78014f03a7235db8d743b3ff, State: Initialized, Role: FOLLOWER
I20260808 00:40:03.620350 26240 consensus_queue.cc:260] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.620728 26240 ts_tablet_manager.cc:1428] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:40:03.620847 26203 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
I20260808 00:40:03.641776 26246 raft_consensus.cc:482] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:03.641918 26246 raft_consensus.cc:504] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.642338 26246 leader_election.cc:288] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009)
I20260808 00:40:03.646054 26226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.643100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:49397 (local address 127.1.208.3:38449)
0808 00:40:03.643159 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.643162 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.643169 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:40:03.643206 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.643207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.643217 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.643266 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:03.643454 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.644077 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.645253 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.645728 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.645765 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.645775 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.645954 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.645991 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:40:03.646257 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.642683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38449 (local address 127.1.208.2:49397)
0808 00:40:03.642847 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.642859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.642920 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.643285 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.643286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.643288 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:03.643378 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.643380 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.644148 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.644149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.645162 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.645164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.645823 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.645824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.645882 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.645886 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.646155 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.646193 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260808 00:40:03.646678 26116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768458a0c6d348ff8d1687f09b8d8eed" candidate_uuid: "571f38c1f2054a5ea94836b0e100e523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" is_pre_election: true
I20260808 00:40:03.646862 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.642494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44009 (local address 127.1.208.2:35517)
0808 00:40:03.642558 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.642570 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.642617 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.642975 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.642976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.642978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:03.643052 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.643055 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.643935 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.643936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.645360 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.645364 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.646614 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.646615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:03.646693 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.646697 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:40:03.646752 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 00:40:03.646795 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:40:03.646865 26223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.642640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:35517 (local address 127.1.208.1:44009)
0808 00:40:03.642834 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.642837 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.642845 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:40:03.642873 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.642874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.642887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.642954 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:03.643126 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.643849 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.645472 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.646165 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.646238 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.646250 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.646780 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.646817 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260808 00:40:03.646885 26116 raft_consensus.cc:2454] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 571f38c1f2054a5ea94836b0e100e523 in term 0.
I20260808 00:40:03.647126 25781 leader_election.cc:302] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff; no voters:
I20260808 00:40:03.647187 25663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768458a0c6d348ff8d1687f09b8d8eed" candidate_uuid: "571f38c1f2054a5ea94836b0e100e523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae824ee7574546c1823995d6be3596ce" is_pre_election: true
I20260808 00:40:03.647333 25663 raft_consensus.cc:2454] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 571f38c1f2054a5ea94836b0e100e523 in term 0.
I20260808 00:40:03.647447 26246 raft_consensus.cc:2781] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:40:03.647513 26246 raft_consensus.cc:482] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:40:03.647536 26246 raft_consensus.cc:3037] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:03.648401 26246 raft_consensus.cc:504] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.648679 26246 leader_election.cc:288] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [CANDIDATE]: Term 1 election: Requested vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009)
I20260808 00:40:03.648705 25663 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768458a0c6d348ff8d1687f09b8d8eed" candidate_uuid: "571f38c1f2054a5ea94836b0e100e523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae824ee7574546c1823995d6be3596ce"
I20260808 00:40:03.648778 25663 raft_consensus.cc:3037] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:03.649376 26116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "768458a0c6d348ff8d1687f09b8d8eed" candidate_uuid: "571f38c1f2054a5ea94836b0e100e523" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
I20260808 00:40:03.649467 26116 raft_consensus.cc:3037] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:03.649718 25663 raft_consensus.cc:2454] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 571f38c1f2054a5ea94836b0e100e523 in term 1.
I20260808 00:40:03.649873 25778 leader_election.cc:302] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571f38c1f2054a5ea94836b0e100e523, ae824ee7574546c1823995d6be3596ce; no voters:
I20260808 00:40:03.649960 26246 raft_consensus.cc:2781] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:03.650076 26246 raft_consensus.cc:686] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 1 LEADER]: Becoming Leader. State: Replica: 571f38c1f2054a5ea94836b0e100e523, State: Running, Role: LEADER
I20260808 00:40:03.650164 26246 consensus_queue.cc:237] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:03.650409 26116 raft_consensus.cc:2454] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 571f38c1f2054a5ea94836b0e100e523 in term 1.
I20260808 00:40:03.651023 25395 catalog_manager.cc:5495] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 reported cstate change: term changed from 0 to 1, leader changed from <none> to 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "571f38c1f2054a5ea94836b0e100e523" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } health_report { overall_health: UNKNOWN } } }
I20260808 00:40:03.666826 26222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.664023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45110 (local address 127.1.208.2:34559)
0808 00:40:03.664070 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.664072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.664128 (+ 56us) server_negotiation.cc:407] Connection header received
0808 00:40:03.664218 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.664219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.664237 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.664299 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:03.664415 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.665060 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.666099 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:03.666303 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:03.666365 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:03.666501 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:03.666607 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:40:03.666678 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:03.666699 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 00:40:03.666719 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:40:03.668844 26116 raft_consensus.cc:1261] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Refusing update from remote peer 571f38c1f2054a5ea94836b0e100e523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:03.668838 25663 raft_consensus.cc:1261] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Refusing update from remote peer 571f38c1f2054a5ea94836b0e100e523: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:03.669185 26246 consensus_queue.cc:1035] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:03.669414 26251 consensus_queue.cc:1035] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:03.671942 26257 mvcc.cc:205] Tried to move back new op lower bound from 7316068776624611328 to 7316068776551735296. Current Snapshot: MvccSnapshot[applied={T|T < 7316068776624611328}]
I20260808 00:40:03.674304 25395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:03.674422 25395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:03.675169 25395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Sending DeleteTablet for 3 replicas of tablet 768458a0c6d348ff8d1687f09b8d8eed
I20260808 00:40:03.675544 26088 tablet_service.cc:1514] Processing DeleteTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:03 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:56910
I20260808 00:40:03.675578 25633 tablet_service.cc:1514] Processing DeleteTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:03 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45232
I20260808 00:40:03.675750 25856 tablet_service.cc:1514] Processing DeleteTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:03 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45104
I20260808 00:40:03.675803 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 25512
I20260808 00:40:03.675981 26271 tablet_replica.cc:330] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: stopping tablet replica
I20260808 00:40:03.676070 26271 raft_consensus.cc:2229] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:03.676151 26271 raft_consensus.cc:2258] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:03.678833 26274 tablet_replica.cc:330] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: stopping tablet replica
I20260808 00:40:03.678967 26274 raft_consensus.cc:2229] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:40:03.679106 26274 raft_consensus.cc:2258] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:03.685655 26271 ts_tablet_manager.cc:1905] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:03.687532 26271 ts_tablet_manager.cc:1918] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:03.687625 26271 log.cc:1196] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/768458a0c6d348ff8d1687f09b8d8eed
I20260808 00:40:03.687927 26271 ts_tablet_manager.cc:1939] T 768458a0c6d348ff8d1687f09b8d8eed P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting consensus metadata
I20260808 00:40:03.688467 25371 catalog_manager.cc:4841] TS 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): tablet 768458a0c6d348ff8d1687f09b8d8eed (table test-table [id=f003beb54b07458786b0ab195259d5de]) successfully deleted
I20260808 00:40:03.693948 26274 ts_tablet_manager.cc:1905] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:03.695515 26274 ts_tablet_manager.cc:1918] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:03.695600 26274 log.cc:1196] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/768458a0c6d348ff8d1687f09b8d8eed
I20260808 00:40:03.695865 26274 ts_tablet_manager.cc:1939] T 768458a0c6d348ff8d1687f09b8d8eed P 571f38c1f2054a5ea94836b0e100e523: Deleting consensus metadata
I20260808 00:40:03.696364 25362 catalog_manager.cc:4841] TS 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): tablet 768458a0c6d348ff8d1687f09b8d8eed (table test-table [id=f003beb54b07458786b0ab195259d5de]) successfully deleted
W20260808 00:40:03.698395 25366 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:44009 (error 108)
I20260808 00:40:03.698588 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 25748
I20260808 00:40:03.701152 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.701013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44009 (local address 127.0.0.1:45246)
0808 00:40:03.701072 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.701118 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 00:40:03.702148 25366 catalog_manager.cc:4584] TS ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 768458a0c6d348ff8d1687f09b8d8eed: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44009: connect: Connection refused (error 111)
I20260808 00:40:03.706382 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 25983
I20260808 00:40:03.716786 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 25331
I20260808 00:40:03.727147 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--webserver_interface=127.1.208.62
--webserver_port=41865
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:03.808987 26277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.809203 26277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.809245 26277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.810447 26277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:03.810491 26277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.810511 26277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.810528 26277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:03.810542 26277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:03.812027 26277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=41865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.812290 26277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.812503 26277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:03.821497 26293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.821501 26290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.821517 26291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.821774 26277 server_base.cc:1034] running on GCE node
I20260808 00:40:03.822064 26277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.822377 26277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.823516 26277 hybrid_clock.cc:648] HybridClock initialized: now 1786149603823500 us; error 29 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.825918 26277 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:03.826334 26277 webserver.cc:460] Webserver started at http://127.1.208.62:41865/ using document root <none> and password file <none>
I20260808 00:40:03.826545 26277 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.826608 26277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.827929 26277 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:03.828513 26301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.828662 26277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:03.828749 26277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "8ce6d00b2d964591a9cc80920ac77d82"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "f7f5064bfbfe1f0bc82857a9aabd4f41"
server_key_iv: "928a0ca7482173d88d7532c1287606f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.829147 26277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.841198 26277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.842859 26277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.842998 26277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.846910 26277 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38045
I20260808 00:40:03.846933 26386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38045 every 8 connection(s)
I20260808 00:40:03.847307 26277 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:03.848930 26388 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:40:03.849606 26388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap starting.
I20260808 00:40:03.852005 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 26277
I20260808 00:40:03.852243 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:44009
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=37105
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:03.858579 26388 log.cc:824] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.860476 26388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:03.860749 26388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap complete.
I20260808 00:40:03.862924 26388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.863145 26388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Initialized, Role: FOLLOWER
I20260808 00:40:03.863288 26388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.863364 26388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:03.863399 26388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:03.863447 26388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:40:03.864420 26388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.864513 26388 leader_election.cc:302] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ce6d00b2d964591a9cc80920ac77d82; no voters:
I20260808 00:40:03.864665 26388 leader_election.cc:288] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:40:03.864718 26394 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:40:03.864846 26394 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 LEADER]: Becoming Leader. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Running, Role: LEADER
I20260808 00:40:03.864940 26388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:03.864950 26394 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:03.865247 26396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:03.865358 26396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:03.865242 26394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ce6d00b2d964591a9cc80920ac77d82. Latest consensus state: current_term: 2 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:03.865533 26394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:03.865862 26403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:40:03.866667 26403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:03.866786 26403 catalog_manager.cc:1259] Loaded cluster ID: 45c8f3f06f1f4fdb9f6478071b9a974d
I20260808 00:40:03.867048 26403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:40:03.867271 26403 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 00:40:03.869989 26403 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 00:40:03.874212 26403 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:40:03.874392 26403 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Loaded TSK: 0
I20260808 00:40:03.874605 26403 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:40:03.946285 26390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:03.946484 26390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:03.946514 26390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:03.947644 26390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:03.947683 26390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:03.947716 26390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:40:03.949172 26390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:44009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=37105
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:03.949436 26390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:03.949640 26390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:40:03.954403 26390 server_base.cc:1034] running on GCE node
W20260808 00:40:03.954387 26435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.954495 26433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:03.954376 26427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:03.954777 26390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:03.955016 26390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:03.956154 26390 hybrid_clock.cc:648] HybridClock initialized: now 1786149603956142 us; error 30 us; skew 500 ppm
Aug 08 00:40:03 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:03.958767 26390 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:40:03.959180 26390 webserver.cc:460] Webserver started at http://127.1.208.1:37105/ using document root <none> and password file <none>
I20260808 00:40:03.959396 26390 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:03.959461 26390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:03.960812 26390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:03.961491 26445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:03.961643 26390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:03.961726 26390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ae824ee7574546c1823995d6be3596ce"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "b690d367b84b30d2ee73ee318b1353d8"
server_key_iv: "a396a00438bbf9842ced7467aa972275"
server_key_version: "encryptionkey@0"
I20260808 00:40:03.962054 26390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:03.981065 26390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:03.983176 26390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:03.983310 26390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:03.983527 26390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:03.983940 26457 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:40:03.985817 26390 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:40:03.985864 26390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:40:03.985906 26390 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:40:03.986538 26390 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:40:03.986569 26390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:03.986651 26457 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Bootstrap starting.
I20260808 00:40:03.993075 26390 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:44009
I20260808 00:40:03.993232 26663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:44009 every 8 connection(s)
I20260808 00:40:03.993542 26390 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:40:03.998605 26457 log.cc:824] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:03.998656 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 26390
I20260808 00:40:03.998941 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:34559
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=37361
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:04.001075 26457 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:04.001412 26457 tablet_bootstrap.cc:492] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Bootstrap complete.
I20260808 00:40:04.002614 26457 ts_tablet_manager.cc:1397] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.010s
I20260808 00:40:04.003842 26457 raft_consensus.cc:348] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.004137 26457 raft_consensus.cc:729] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae824ee7574546c1823995d6be3596ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.004289 26457 consensus_queue.cc:260] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.004851 26457 ts_tablet_manager.cc:1428] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:04.010078 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.994231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:53169)
0808 00:40:03.997503 (+ 3272us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:03.997539 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:03.998038 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:03.998453 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:03.998457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:03.998830 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:03.999103 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:03.999111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.002293 (+ 3182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.002294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.003322 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.003336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.004070 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.004071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.004203 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.004790 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:04.004800 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:04.006533 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:04.008569 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.008575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.008577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.008788 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.009044 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.009045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.009045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.009132 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.009599 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:04.009603 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:04.009715 (+ 112us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.009797 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.009932 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3237,"thread_start_us":121,"threads_started":1}
I20260808 00:40:04.010231 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:03.994333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:53169 (local address 127.1.208.62:38045)
0808 00:40:03.994535 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:03.994543 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:03.997654 (+ 3111us) server_negotiation.cc:407] Connection header received
0808 00:40:03.998240 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:03.998246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:03.998291 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:03.998410 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:04.000861 (+ 2451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.002182 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.003431 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.003968 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.004565 (+ 597us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.006641 (+ 2076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:04.006647 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:04.006647 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:04.006660 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:04.008480 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.008862 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.008865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.008866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.008982 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.009189 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.009190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.009190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.009536 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:04.009627 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.009879 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.010036 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260808 00:40:04.010983 26664 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:04.011075 26664 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:04.011245 26664 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:04.011917 26333 ts_manager.cc:194] Registered new tserver with Master: ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009)
I20260808 00:40:04.012965 26333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:53169
I20260808 00:40:04.013877 26664 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149603, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:40:04.020443 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.012673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45260 (local address 127.1.208.1:44009)
0808 00:40:04.013043 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.013048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.013059 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:40:04.013092 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.013094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.013107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.013181 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:04.013788 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.014610 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.015455 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.015837 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.015913 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.017923 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:04.017930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:04.017931 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:04.017938 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:04.019319 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.019669 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.019671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.019672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.019724 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.019890 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.019891 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.019891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.020022 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:04.020090 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.020261 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.020346 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":66,"threads_started":1}
I20260808 00:40:04.020440 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.012477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44009 (local address 127.0.0.1:45260)
0808 00:40:04.012642 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.012681 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.012750 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.013325 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.013328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.013646 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:04.013699 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.013701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.014681 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.014682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.015386 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.015390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.015939 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.015941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.016028 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.016405 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:04.016409 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:04.017842 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:04.019397 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.019405 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.019407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.019594 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.019778 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.019779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.019780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.019840 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.020082 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:04.020089 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:04.020144 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.020217 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.020336 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260808 00:40:04.020884 26537 tablet_service.cc:1514] Processing DeleteTablet for tablet 768458a0c6d348ff8d1687f09b8d8eed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:03 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45260
I20260808 00:40:04.021485 26682 tablet_replica.cc:330] stopping tablet replica
I20260808 00:40:04.021584 26682 raft_consensus.cc:2229] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:04.021649 26682 raft_consensus.cc:2258] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:04.021927 26682 ts_tablet_manager.cc:1905] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:04.023947 26682 ts_tablet_manager.cc:1918] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:04.024016 26682 log.cc:1196] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/768458a0c6d348ff8d1687f09b8d8eed
I20260808 00:40:04.024197 26682 ts_tablet_manager.cc:1939] T 768458a0c6d348ff8d1687f09b8d8eed P ae824ee7574546c1823995d6be3596ce: Deleting consensus metadata
I20260808 00:40:04.024645 26306 catalog_manager.cc:4841] TS ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009): tablet 768458a0c6d348ff8d1687f09b8d8eed (table test-table [id=f003beb54b07458786b0ab195259d5de]) successfully deleted
W20260808 00:40:04.088547 26674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.088759 26674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.088781 26674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.089937 26674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.089978 26674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.090006 26674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:40:04.091418 26674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:34559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=37361
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.091625 26674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.091861 26674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.096742 26702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.096797 26697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.096755 26696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.096974 26674 server_base.cc:1034] running on GCE node
I20260808 00:40:04.097204 26674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.097468 26674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.098618 26674 hybrid_clock.cc:648] HybridClock initialized: now 1786149604098599 us; error 28 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.100971 26674 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:40:04.101428 26674 webserver.cc:460] Webserver started at http://127.1.208.2:37361/ using document root <none> and password file <none>
I20260808 00:40:04.101624 26674 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.101688 26674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.102964 26674 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:40:04.103471 26711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.103623 26674 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:40:04.103706 26674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "571f38c1f2054a5ea94836b0e100e523"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "2e4838c5067ec05c23a7b4145a93e7c3"
server_key_iv: "85192ccd48087d5bd15a947f9cfe8ead"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.104038 26674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.120095 26674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.122229 26674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.122365 26674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.122596 26674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:04.122889 26674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:04.122925 26674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.122956 26674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:04.122978 26674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.128937 26674 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:34559
I20260808 00:40:04.128975 26908 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:34559 every 8 connection(s)
I20260808 00:40:04.129325 26674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:04.136487 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 26674
I20260808 00:40:04.136730 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:38449
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=35489
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:04.138293 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.130018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:47383 (local address 127.1.208.62:38045)
0808 00:40:04.130076 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.130078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.130258 (+ 180us) server_negotiation.cc:407] Connection header received
0808 00:40:04.130784 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.130785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.130799 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.130851 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:04.131839 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.132339 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.133451 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.133665 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.133722 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.135630 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:04.135631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:04.135631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:04.135639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:04.136794 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.137384 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.137386 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.137386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.137456 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.137697 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.137698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.137698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.137836 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:04.137897 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.138116 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.138202 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:40:04.138459 26911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.129997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:47383)
0808 00:40:04.130144 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.130181 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.130684 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.130924 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.130927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.131514 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:04.131731 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.131738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.132433 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.132434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.133373 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.133376 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.133734 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.133734 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.133853 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.134278 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:04.134286 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:04.135542 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:04.136881 (+ 1339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.136888 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.136890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.137258 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.137526 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.137527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.137527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.137637 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.137891 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:04.137894 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:04.138001 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.138089 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.138257 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260808 00:40:04.138983 26909 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:04.139087 26909 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:04.139266 26909 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:04.139660 26333 ts_manager.cc:194] Registered new tserver with Master: 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
I20260808 00:40:04.140050 26333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:47383
W20260808 00:40:04.219183 26912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.219379 26912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.219400 26912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.220503 26912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.220542 26912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.220571 26912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:40:04.222012 26912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:38449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=35489
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.222234 26912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.222461 26912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.227167 26930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.227200 26925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.227320 26912 server_base.cc:1034] running on GCE node
W20260808 00:40:04.227435 26924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.227622 26912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.227857 26912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.228998 26912 hybrid_clock.cc:648] HybridClock initialized: now 1786149604228987 us; error 25 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.231492 26912 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:40:04.231923 26912 webserver.cc:460] Webserver started at http://127.1.208.3:35489/ using document root <none> and password file <none>
I20260808 00:40:04.232157 26912 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.232223 26912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.233534 26912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:04.234061 26940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.234210 26912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.234292 26912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "eb0406fafad5fc932797cb69834266c8"
server_key_iv: "e3b0a630f119ed8b1a3ccbb8901c108c"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.234635 26912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.257145 26912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.259099 26912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.259238 26912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.259472 26912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:04.259781 26912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:04.259816 26912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.259846 26912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:04.259863 26912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.265851 26912 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:38449
I20260808 00:40:04.265897 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:38449 every 8 connection(s)
I20260808 00:40:04.266230 26912 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:04.272819 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 26912
I20260808 00:40:04.274825 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.266918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:51873 (local address 127.1.208.62:38045)
0808 00:40:04.266966 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.266968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.267174 (+ 206us) server_negotiation.cc:407] Connection header received
0808 00:40:04.267652 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.267653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.267664 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.267711 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:04.268591 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.269076 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.270025 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.270214 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.270260 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.272144 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:04.272146 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:04.272146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:04.272153 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:04.273511 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.273961 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.273961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.273962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.274016 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.274271 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.274271 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.274271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.274371 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:04.274415 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.274702 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.274760 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:40:04.275087 27141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.266880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:51873)
0808 00:40:04.267041 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.267078 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.267570 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.267767 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.267770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.268318 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:04.268507 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.268511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.269139 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.269140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.269957 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.269960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.270266 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.270267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.270356 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.270758 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:04.270766 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:04.272062 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:04.273579 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.273587 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.273589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.273893 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.274069 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:04.274070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:04.274070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:04.274210 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:04.274420 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:04.274425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:04.274577 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.274679 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.274851 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":97,"threads_started":1}
I20260808 00:40:04.275519 27138 heartbeater.cc:347] Connected to a master server at 127.1.208.62:38045
I20260808 00:40:04.275614 27138 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:04.275816 27138 heartbeater.cc:511] Master 127.1.208.62:38045 requested a full tablet report, sending...
I20260808 00:40:04.276253 26333 ts_manager.cc:194] Registered new tserver with Master: 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449)
I20260808 00:40:04.276695 26333 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:51873
I20260808 00:40:04.279513 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.273172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.1.208.62:38045)
0808 00:40:04.273334 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.273335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.273357 (+ 22us) server_negotiation.cc:407] Connection header received
0808 00:40:04.273378 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.273378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.273389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.273440 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:04.274104 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.274799 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.275760 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.276053 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.276118 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.277436 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:04.277437 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:04.277437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:04.277444 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:04.278706 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.278962 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.278963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.278963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.279008 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:04.279127 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:04.279128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:04.279128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:04.279212 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:04.279341 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.279368 (+ 27us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.279426 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260808 00:40:04.280153 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:40:04.282498 27144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.280469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.1.208.62:38045)
0808 00:40:04.280506 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.280508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.280607 (+ 99us) server_negotiation.cc:407] Connection header received
0808 00:40:04.280664 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.280664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.280694 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.280745 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:04.280816 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.281236 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.281938 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.282115 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.282155 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.282261 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:04.282357 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:40:04.282409 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.282425 (+ 16us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.282440 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:40:04.282758 26333 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:49264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:40:04.284302 26333 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:40:04.286243 26537 tablet_service.cc:1467] Processing CreateTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 (DEFAULT_TABLE table=test-table [id=f31ed54d977d49a5b6e8a1ee94622ca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:04.286453 26537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7264cac913c0431f8f338cdd6ada3d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:04.287868 26457 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Bootstrap starting.
I20260808 00:40:04.288182 26457 tablet_bootstrap.cc:654] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:04.288661 26457 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: No bootstrap required, opened a new log
I20260808 00:40:04.288704 26457 ts_tablet_manager.cc:1397] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:04.289033 26457 raft_consensus.cc:348] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.289098 26457 raft_consensus.cc:374] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:04.289147 26457 raft_consensus.cc:729] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae824ee7574546c1823995d6be3596ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.289191 26457 consensus_queue.cc:260] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.289279 26457 ts_tablet_manager.cc:1428] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:04.290411 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.286042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56914 (local address 127.1.208.3:38449)
0808 00:40:04.286272 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.286277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.286286 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:40:04.286323 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.286325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.286339 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.286609 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:04.286758 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.287870 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.289268 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.290012 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.290080 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.290138 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.290291 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.290336 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260808 00:40:04.290412 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.286223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45126 (local address 127.1.208.2:34559)
0808 00:40:04.286438 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.286443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.286535 (+ 92us) server_negotiation.cc:407] Connection header received
0808 00:40:04.286596 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.286598 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.286615 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.286688 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:04.286801 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.287884 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.289265 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.290012 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.290080 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.290137 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.290291 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.290330 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
I20260808 00:40:04.290441 26679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.285966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38449 (local address 127.0.0.1:56914)
0808 00:40:04.286007 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.286017 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.286050 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.286617 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.286618 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.286620 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:04.286680 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.286682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.288062 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.288063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.289171 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.289173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.290101 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.290102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.290199 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.290214 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.290276 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.290326 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":20}
I20260808 00:40:04.290441 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.286010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:34559 (local address 127.0.0.1:45126)
0808 00:40:04.286465 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.286473 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.286527 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.286701 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.286702 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.286704 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:04.286746 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.286747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.287993 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.287995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.289170 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.289173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.290102 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.290103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.290191 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.290216 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.290276 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.290324 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":8,"thread_start_us":113,"threads_started":1}
I20260808 00:40:04.291162 27027 tablet_service.cc:1467] Processing CreateTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 (DEFAULT_TABLE table=test-table [id=f31ed54d977d49a5b6e8a1ee94622ca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:04.291446 27027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7264cac913c0431f8f338cdd6ada3d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:04.291450 26790 tablet_service.cc:1467] Processing CreateTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 (DEFAULT_TABLE table=test-table [id=f31ed54d977d49a5b6e8a1ee94622ca9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:04.291769 26790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7264cac913c0431f8f338cdd6ada3d37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:04.294307 27157 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Bootstrap starting.
I20260808 00:40:04.294502 27160 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Bootstrap starting.
I20260808 00:40:04.295090 27157 tablet_bootstrap.cc:654] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:04.295121 27160 tablet_bootstrap.cc:654] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:04.295504 27157 log.cc:824] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:04.295553 27160 log.cc:824] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:04.296166 27157 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: No bootstrap required, opened a new log
I20260808 00:40:04.296222 27157 ts_tablet_manager.cc:1397] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:40:04.296262 27160 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: No bootstrap required, opened a new log
I20260808 00:40:04.296320 27160 ts_tablet_manager.cc:1397] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:40:04.297581 27157 raft_consensus.cc:348] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.297721 27157 raft_consensus.cc:374] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:04.297755 27157 raft_consensus.cc:729] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c6bb9ef78014f03a7235db8d743b3ff, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.297704 27160 raft_consensus.cc:348] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.297857 27160 raft_consensus.cc:374] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:04.297904 27160 raft_consensus.cc:729] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571f38c1f2054a5ea94836b0e100e523, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.297873 27157 consensus_queue.cc:260] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.298015 27160 consensus_queue.cc:260] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.298149 27157 ts_tablet_manager.cc:1428] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:40:04.298170 27138 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
I20260808 00:40:04.298288 27160 ts_tablet_manager.cc:1428] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:40:04.298425 26909 heartbeater.cc:503] Master 127.1.208.62:38045 was elected leader, sending a full tablet report...
I20260808 00:40:04.337658 26678 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:04.337811 26678 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.338229 26678 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
I20260808 00:40:04.341027 27153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.338475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60227 (local address 127.1.208.2:34559)
0808 00:40:04.338539 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.338541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.338551 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:40:04.338614 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.338615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.338627 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.338688 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:04.338850 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.339346 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340317 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.340678 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340729 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.340741 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.340902 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.340958 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 00:40:04.341042 27148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.338596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:43137 (local address 127.1.208.3:38449)
0808 00:40:04.338644 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.338646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.338670 (+ 24us) server_negotiation.cc:407] Connection header received
0808 00:40:04.338746 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.338746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.338759 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.338816 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:04.338971 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.339436 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340315 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.340694 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340759 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.340768 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.340930 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.340983 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:40:04.341035 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.338399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:34559 (local address 127.1.208.1:60227)
0808 00:40:04.338470 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.338483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.338539 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.338698 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.338699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.338701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:04.338783 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.338785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.339425 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.339426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.340232 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.340235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.340749 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.340821 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.340843 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.340885 (+ 42us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.340927 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":28}
I20260808 00:40:04.341116 27169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.338408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38449 (local address 127.1.208.1:43137)
0808 00:40:04.338620 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.338629 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.338684 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.338823 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:04.338824 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:04.338826 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:04.338909 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.338911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.339504 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.339504 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.340235 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:04.340237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.340791 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.340793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:04.340856 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.340859 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:40:04.340910 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:40:04.340948 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":24,"thread_start_us":108,"threads_started":1}
I20260808 00:40:04.341420 26826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7264cac913c0431f8f338cdd6ada3d37" candidate_uuid: "ae824ee7574546c1823995d6be3596ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571f38c1f2054a5ea94836b0e100e523" is_pre_election: true
I20260808 00:40:04.341580 26826 raft_consensus.cc:2454] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae824ee7574546c1823995d6be3596ce in term 0.
I20260808 00:40:04.341599 27061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7264cac913c0431f8f338cdd6ada3d37" candidate_uuid: "ae824ee7574546c1823995d6be3596ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" is_pre_election: true
I20260808 00:40:04.341755 27061 raft_consensus.cc:2454] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae824ee7574546c1823995d6be3596ce in term 0.
I20260808 00:40:04.341809 26446 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571f38c1f2054a5ea94836b0e100e523, ae824ee7574546c1823995d6be3596ce; no voters:
I20260808 00:40:04.341940 26678 raft_consensus.cc:2781] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:40:04.341990 26678 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:40:04.342018 26678 raft_consensus.cc:3037] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:04.342677 26678 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.342801 26678 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 1 election: Requested vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
I20260808 00:40:04.342931 26826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7264cac913c0431f8f338cdd6ada3d37" candidate_uuid: "ae824ee7574546c1823995d6be3596ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "571f38c1f2054a5ea94836b0e100e523"
I20260808 00:40:04.342988 27061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7264cac913c0431f8f338cdd6ada3d37" candidate_uuid: "ae824ee7574546c1823995d6be3596ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
I20260808 00:40:04.343012 26826 raft_consensus.cc:3037] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:04.343056 27061 raft_consensus.cc:3037] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:04.343842 26826 raft_consensus.cc:2454] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae824ee7574546c1823995d6be3596ce in term 1.
I20260808 00:40:04.343858 27061 raft_consensus.cc:2454] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae824ee7574546c1823995d6be3596ce in term 1.
I20260808 00:40:04.344007 26446 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 571f38c1f2054a5ea94836b0e100e523, ae824ee7574546c1823995d6be3596ce; no voters:
I20260808 00:40:04.344115 26678 raft_consensus.cc:2781] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:04.344278 26678 raft_consensus.cc:686] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 LEADER]: Becoming Leader. State: Replica: ae824ee7574546c1823995d6be3596ce, State: Running, Role: LEADER
I20260808 00:40:04.344338 26678 consensus_queue.cc:237] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.345095 26332 catalog_manager.cc:5495] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce reported cstate change: term changed from 0 to 1, leader changed from <none> to ae824ee7574546c1823995d6be3596ce (127.1.208.1). New cstate: current_term: 1 leader_uuid: "ae824ee7574546c1823995d6be3596ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } health_report { overall_health: HEALTHY } } }
I20260808 00:40:04.362771 26680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.360779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.1.208.1:44009)
0808 00:40:04.360825 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:04.360827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:04.360836 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:40:04.360867 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:04.360868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:04.360884 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:04.360938 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:04.361027 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.361546 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.362190 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:04.362387 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:04.362429 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:04.362530 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:04.362625 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:40:04.362671 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:04.362700 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 00:40:04.362717 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:40:04.364249 27061 raft_consensus.cc:1261] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Refusing update from remote peer ae824ee7574546c1823995d6be3596ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:04.364271 26826 raft_consensus.cc:1261] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Refusing update from remote peer ae824ee7574546c1823995d6be3596ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:04.364547 26678 consensus_queue.cc:1035] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:04.364722 27170 consensus_queue.cc:1035] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:04.367367 27174 mvcc.cc:205] Tried to move back new op lower bound from 7316068779473682432 to 7316068779394961408. Current Snapshot: MvccSnapshot[applied={T|T < 7316068779473682432}]
I20260808 00:40:04.367501 27173 mvcc.cc:205] Tried to move back new op lower bound from 7316068779473682432 to 7316068779394961408. Current Snapshot: MvccSnapshot[applied={T|T < 7316068779473682432}]
I20260808 00:40:04.369352 26332 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:49264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:04.369436 26332 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:49264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:04.370074 26332 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Sending DeleteTablet for 3 replicas of tablet 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:04.370393 26537 tablet_service.cc:1514] Processing DeleteTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45260
I20260808 00:40:04.370416 27027 tablet_service.cc:1514] Processing DeleteTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:56914
I20260808 00:40:04.370515 26682 tablet_replica.cc:330] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: stopping tablet replica
I20260808 00:40:04.370579 26682 raft_consensus.cc:2229] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:40:04.370628 26790 tablet_service.cc:1514] Processing DeleteTablet for tablet 7264cac913c0431f8f338cdd6ada3d37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:45126
I20260808 00:40:04.370661 26682 raft_consensus.cc:2258] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:04.370847 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 26390
I20260808 00:40:04.370929 26682 ts_tablet_manager.cc:1905] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:04.379817 27187 tablet_replica.cc:330] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: stopping tablet replica
I20260808 00:40:04.380097 27187 raft_consensus.cc:2229] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:04.380268 27187 raft_consensus.cc:2258] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:04.383255 27184 tablet_replica.cc:330] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: stopping tablet replica
I20260808 00:40:04.383391 27184 raft_consensus.cc:2229] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:04.383551 27184 raft_consensus.cc:2258] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:04.384536 27184 ts_tablet_manager.cc:1905] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:04.384547 27187 ts_tablet_manager.cc:1905] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:04.386062 27187 ts_tablet_manager.cc:1918] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:04.386071 27184 ts_tablet_manager.cc:1918] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:04.386133 27184 log.cc:1196] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:04.386133 27187 log.cc:1196] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:04.386389 27187 ts_tablet_manager.cc:1939] T 7264cac913c0431f8f338cdd6ada3d37 P 571f38c1f2054a5ea94836b0e100e523: Deleting consensus metadata
I20260808 00:40:04.386415 27184 ts_tablet_manager.cc:1939] T 7264cac913c0431f8f338cdd6ada3d37 P 8c6bb9ef78014f03a7235db8d743b3ff: Deleting consensus metadata
I20260808 00:40:04.386801 26302 catalog_manager.cc:4841] TS 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): tablet 7264cac913c0431f8f338cdd6ada3d37 (table test-table [id=f31ed54d977d49a5b6e8a1ee94622ca9]) successfully deleted
I20260808 00:40:04.386914 26312 catalog_manager.cc:4841] TS 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): tablet 7264cac913c0431f8f338cdd6ada3d37 (table test-table [id=f31ed54d977d49a5b6e8a1ee94622ca9]) successfully deleted
W20260808 00:40:04.389026 26306 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:44009 (error 108)
I20260808 00:40:04.389410 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 26674
I20260808 00:40:04.389489 27151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.389369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44009 (local address 127.0.0.1:45278)
0808 00:40:04.389419 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.389457 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:40:04.411602 26306 catalog_manager.cc:4584] TS ae824ee7574546c1823995d6be3596ce (127.1.208.1:44009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7264cac913c0431f8f338cdd6ada3d37: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44009: connect: Connection refused (error 111)
I20260808 00:40:04.417126 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 26912
I20260808 00:40:04.425995 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 26277
I20260808 00:40:04.436194 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--webserver_interface=127.1.208.62
--webserver_port=41865
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38045
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:04.519143 27192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.519359 27192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.519390 27192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.520509 27192 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:04.520597 27192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.520670 27192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.520710 27192 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:04.520736 27192 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:04.522406 27192 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38045
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=41865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.522663 27192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.522876 27192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.527587 27202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.527720 27203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.527596 27207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.527828 27192 server_base.cc:1034] running on GCE node
I20260808 00:40:04.528116 27192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.528419 27192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.529553 27192 hybrid_clock.cc:648] HybridClock initialized: now 1786149604529536 us; error 32 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.531919 27192 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:04.532295 27192 webserver.cc:460] Webserver started at http://127.1.208.62:41865/ using document root <none> and password file <none>
I20260808 00:40:04.532480 27192 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.532522 27192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.533799 27192 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:04.534359 27218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.534518 27192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:04.534601 27192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "8ce6d00b2d964591a9cc80920ac77d82"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "f7f5064bfbfe1f0bc82857a9aabd4f41"
server_key_iv: "928a0ca7482173d88d7532c1287606f4"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.534911 27192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.545411 27192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.546983 27192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.547083 27192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.550844 27192 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38045
I20260808 00:40:04.550882 27303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38045 every 8 connection(s)
I20260808 00:40:04.551208 27192 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:04.552735 27304 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:40:04.553416 27304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap starting.
I20260808 00:40:04.560927 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 27192
I20260808 00:40:04.561187 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:44009
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=37105
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:04.562219 27304 log.cc:824] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:04.564603 27304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:04.564915 27304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82: Bootstrap complete.
I20260808 00:40:04.568092 27304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:04.568431 27304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.568612 27304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:04.568691 27304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:04.568742 27304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:04.568792 27304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:40:04.570214 27304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:04.570345 27304 leader_election.cc:302] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ce6d00b2d964591a9cc80920ac77d82; no voters:
I20260808 00:40:04.570535 27304 leader_election.cc:288] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:40:04.570590 27313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:40:04.570741 27313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [term 3 LEADER]: Becoming Leader. State: Replica: 8ce6d00b2d964591a9cc80920ac77d82, State: Running, Role: LEADER
I20260808 00:40:04.570811 27304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:04.570852 27313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } }
I20260808 00:40:04.571101 27313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ce6d00b2d964591a9cc80920ac77d82. Latest consensus state: current_term: 3 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:04.571223 27313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:04.571422 27325 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:40:04.571103 27315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8ce6d00b2d964591a9cc80920ac77d82" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ce6d00b2d964591a9cc80920ac77d82" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38045 } } }
I20260808 00:40:04.571563 27315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ce6d00b2d964591a9cc80920ac77d82 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:04.572366 27325 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:04.572520 27325 catalog_manager.cc:1259] Loaded cluster ID: 45c8f3f06f1f4fdb9f6478071b9a974d
I20260808 00:40:04.572721 27325 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 00:40:04.573027 27325 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f81606089ad google::LogMessage::Fail() at ??:0
@ 0x7f816060b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f81606084de google::LogMessage::Flush() at ??:0
@ 0x7f8160609a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f81685290f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f81685328b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f81624d60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f81624cb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f816332f6db start_thread at ??:0
@ 0x7f815f10661f clone at ??:0
W20260808 00:40:04.659998 27311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.660193 27311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.660218 27311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.661369 27311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.661410 27311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.661435 27311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:40:04.662899 27311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:44009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=37105
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.663127 27311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.663353 27311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.669209 27367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.669243 27364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.669350 27363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.669845 27311 server_base.cc:1034] running on GCE node
I20260808 00:40:04.670024 27311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.670244 27311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.671398 27311 hybrid_clock.cc:648] HybridClock initialized: now 1786149604671375 us; error 40 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.673868 27311 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:40:04.674320 27311 webserver.cc:460] Webserver started at http://127.1.208.1:37105/ using document root <none> and password file <none>
I20260808 00:40:04.674533 27311 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.674576 27311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.675854 27311 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:04.676543 27379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.676720 27311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:04.676831 27311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ae824ee7574546c1823995d6be3596ce"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "b690d367b84b30d2ee73ee318b1353d8"
server_key_iv: "a396a00438bbf9842ced7467aa972275"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.677281 27311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.697916 27311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.700353 27311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.700537 27311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.700811 27311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:04.701352 27392 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:40:04.702790 27311 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:40:04.702852 27311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:40:04.702884 27311 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:40:04.703790 27311 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:40:04.703838 27311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:04.703902 27392 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Bootstrap starting.
I20260808 00:40:04.705467 27392 log.cc:824] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:04.708442 27392 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:04.708758 27392 tablet_bootstrap.cc:492] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Bootstrap complete.
I20260808 00:40:04.708961 27392 ts_tablet_manager.cc:1397] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.003s
I20260808 00:40:04.710731 27392 raft_consensus.cc:348] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.711046 27392 raft_consensus.cc:729] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae824ee7574546c1823995d6be3596ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:04.711243 27392 consensus_queue.cc:260] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:04.711298 27311 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:44009
I20260808 00:40:04.711469 27576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:44009 every 8 connection(s)
I20260808 00:40:04.711612 27392 ts_tablet_manager.cc:1428] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:40:04.711759 27311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:40:04.717942 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 27311
I20260808 00:40:04.718186 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:34559
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=37361
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:04.763314 27583 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:40:04.712508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:49311)
0808 00:40:04.712675 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.712706 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:04.712835 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:04.763084 (+ 50249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
W20260808 00:40:04.763574 27579 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:38045 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:38045: Client connection negotiation failed: client connection to 127.1.208.62:38045: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:40:04.763900 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.763791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:43777)
0808 00:40:04.763838 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.763877 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:40:04.764225 27583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.764135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:41425)
0808 00:40:04.764177 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.764202 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:40:04.764355 27579 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:40:04.803196 27584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.803406 27584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.803428 27584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.804486 27584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.804517 27584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.804539 27584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:40:04.806027 27584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:34559
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=37361
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.806222 27584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.806406 27584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.811123 27592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.811156 27591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.811344 27584 server_base.cc:1034] running on GCE node
W20260808 00:40:04.811349 27595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.811565 27584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.811792 27584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.812939 27584 hybrid_clock.cc:648] HybridClock initialized: now 1786149604812925 us; error 27 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.815358 27584 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:40:04.815775 27584 webserver.cc:460] Webserver started at http://127.1.208.2:37361/ using document root <none> and password file <none>
I20260808 00:40:04.815994 27584 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.816045 27584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.817373 27584 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:40:04.817916 27608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.818059 27584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:04.818142 27584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "571f38c1f2054a5ea94836b0e100e523"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "2e4838c5067ec05c23a7b4145a93e7c3"
server_key_iv: "85192ccd48087d5bd15a947f9cfe8ead"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.818475 27584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.841825 27584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.843891 27584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.844027 27584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.844237 27584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:04.844543 27584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:04.844576 27584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.844605 27584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:04.844627 27584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.850340 27584 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:34559
I20260808 00:40:04.850394 27806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:34559 every 8 connection(s)
I20260808 00:40:04.850714 27584 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:40:04.851679 27812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.851313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:55591)
0808 00:40:04.851468 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.851536 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260808 00:40:04.852041 27808 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:38045 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:38045: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
I20260808 00:40:04.852278 27812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.852199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:57545)
0808 00:40:04.852235 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.852260 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 00:40:04.852483 27812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.852424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:45103)
0808 00:40:04.852452 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.852469 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:40:04.852576 27808 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:40:04.854336 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 27584
I20260808 00:40:04.854562 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:38449
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:38045
--webserver_port=35489
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:04.936911 27813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:04.937107 27813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:04.937150 27813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:04.938319 27813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:04.938370 27813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:04.938403 27813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:40:04.939857 27813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:38449
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=35489
--tserver_master_addrs=127.1.208.62:38045
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:04.940047 27813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:04.940231 27813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:04.944887 27823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.944918 27822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:04.944921 27827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:04.945019 27813 server_base.cc:1034] running on GCE node
I20260808 00:40:04.945353 27813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:04.945571 27813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:04.946703 27813 hybrid_clock.cc:648] HybridClock initialized: now 1786149604946688 us; error 29 us; skew 500 ppm
Aug 08 00:40:04 dist-test-slave-gjw7 krb5kdc[25311](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:04.949014 27813 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:40:04.949481 27813 webserver.cc:460] Webserver started at http://127.1.208.3:35489/ using document root <none> and password file <none>
I20260808 00:40:04.949678 27813 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:04.949741 27813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:04.950978 27813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:04.951479 27842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.951628 27813 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.951707 27813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "8c6bb9ef78014f03a7235db8d743b3ff"
format_stamp: "Formatted at 2026-08-08 00:40:03 on dist-test-slave-gjw7"
server_key: "eb0406fafad5fc932797cb69834266c8"
server_key_iv: "e3b0a630f119ed8b1a3ccbb8901c108c"
server_key_version: "encryptionkey@0"
I20260808 00:40:04.952025 27813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:04.962982 27813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:04.964602 27813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:04.964704 27813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:04.964884 27813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:04.965128 27813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:04.965157 27813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.965176 27813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:04.965193 27813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:04.970844 27813 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:38449
I20260808 00:40:04.970878 28044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:38449 every 8 connection(s)
I20260808 00:40:04.971194 27813 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:40:04.972196 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.971826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:41963)
0808 00:40:04.971969 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.972036 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
W20260808 00:40:04.972553 28045 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:38045 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:38045: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
I20260808 00:40:04.972800 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.972720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:58475)
0808 00:40:04.972756 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.972783 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260808 00:40:04.973026 28047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:04.972967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:51935)
0808 00:40:04.972994 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:04.973013 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260808 00:40:04.973109 28045 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:40:04.980181 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 27813
I20260808 00:40:05.764976 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:05.764732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:54783)
0808 00:40:05.764873 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:05.764911 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260808 00:40:05.853173 28055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:05.852942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:43281)
0808 00:40:05.853077 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:05.853116 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
I20260808 00:40:05.973668 28057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:05.973432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:41667)
0808 00:40:05.973571 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:05.973608 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260808 00:40:06.253891 28060 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:06.254029 28060 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:06.254495 28060 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
Aug 08 00:40:06 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.3@KRBTEST.COM
Aug 08 00:40:06 dist-test-slave-gjw7 krb5kdc[25311](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149604, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.2@KRBTEST.COM
I20260808 00:40:06.264670 28062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.254795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:57405 (local address 127.1.208.2:34559)
0808 00:40:06.254974 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:06.254989 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:06.255025 (+ 36us) server_negotiation.cc:407] Connection header received
0808 00:40:06.255525 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:06.255527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:06.255565 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:06.255688 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:06.256523 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.257504 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.258532 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.258923 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.259016 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:06.261600 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:06.261605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:06.261605 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:06.261617 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:06.263070 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263513 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:06.263516 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:06.263517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:06.263609 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263842 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:06.263843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:06.263843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:06.264083 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:06.264152 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:06.264428 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 00:40:06.264553 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260808 00:40:06.264683 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.254882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:38295 (local address 127.1.208.3:38449)
0808 00:40:06.255057 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:06.255065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:06.255093 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:40:06.255517 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:06.255519 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:06.255560 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:06.255688 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:06.256694 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.257608 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.258526 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.258920 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.259016 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:06.261192 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:06.261195 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:06.261196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:06.261206 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:06.263014 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263525 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:06.263528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:06.263529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:06.263622 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263846 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:06.263847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:06.263847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:06.264084 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:06.264154 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:06.264435 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 00:40:06.264566 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
I20260808 00:40:06.264685 28052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.254646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:38449 (local address 127.1.208.1:38295)
0808 00:40:06.254718 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:06.254729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:06.254786 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:06.255748 (+ 962us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:06.255751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:06.256144 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:06.256585 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:06.256588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.257712 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.257713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:06.258447 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:06.258450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.259002 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.259003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:06.259121 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:06.259706 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:06.259710 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:06.261089 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:06.263096 (+ 2007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263105 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:06.263107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:06.263441 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:06.263687 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:06.263689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:06.263784 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:06.264149 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:06.264155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:06.264275 (+ 120us) client_negotiation.cc:769] Sending connection context
0808 00:40:06.264355 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:40:06.264548 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":21}
I20260808 00:40:06.264775 28061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.254646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:34559 (local address 127.1.208.1:57405)
0808 00:40:06.254869 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:06.254881 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:06.254947 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:06.255751 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:06.255752 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:06.256147 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:06.256398 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:06.256405 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.257593 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.257595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:06.258448 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:06.258451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:06.259003 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:06.259005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:06.259121 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:06.259660 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:06.259668 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:06.261499 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:06.263147 (+ 1648us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:06.263149 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:06.263440 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:06.263669 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:06.263670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:06.263671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:06.263782 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:06.264148 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:06.264155 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:06.264268 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:40:06.264356 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:40:06.264621 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":203,"thread_start_us":69,"threads_started":1}
W20260808 00:40:06.265626 27385 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
W20260808 00:40:06.266201 27380 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:06.266252 27380 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae824ee7574546c1823995d6be3596ce; no voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff
I20260808 00:40:06.266387 28060 raft_consensus.cc:2726] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:40:06.765653 28064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.765437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:39633)
0808 00:40:06.765555 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:06.765600 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":54,"threads_started":1}
I20260808 00:40:06.853816 28065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.853613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:57947)
0808 00:40:06.853732 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:06.853770 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260808 00:40:06.974249 28067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:06.974047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:34533)
0808 00:40:06.974173 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:06.974210 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260808 00:40:07.766297 28071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:07.766056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:58939)
0808 00:40:07.766200 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:07.766238 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260808 00:40:07.854408 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:07.854203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:38875)
0808 00:40:07.854314 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:07.854365 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":63,"threads_started":1}
I20260808 00:40:07.974812 28073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:07.974623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:41215)
0808 00:40:07.974738 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:07.974770 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":64,"threads_started":1}
I20260808 00:40:08.165658 28074 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:08.165758 28074 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:08.165933 28074 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
W20260808 00:40:08.166342 27385 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
W20260808 00:40:08.166433 27380 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:08.166471 27380 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae824ee7574546c1823995d6be3596ce; no voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff
I20260808 00:40:08.166555 28074 raft_consensus.cc:2726] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:40:08.766978 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:08.766753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:51205)
0808 00:40:08.766892 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:08.766929 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260808 00:40:08.854975 28078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:08.854752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:56229)
0808 00:40:08.854893 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:08.854931 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":80,"threads_started":1}
I20260808 00:40:08.975418 28079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:08.975189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:58935)
0808 00:40:08.975338 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:08.975374 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260808 00:40:09.767686 28080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:09.767419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:49601)
0808 00:40:09.767595 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:09.767637 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":89,"threads_started":1}
I20260808 00:40:09.855571 28083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:09.855343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:33169)
0808 00:40:09.855489 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:09.855527 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":90,"threads_started":1}
I20260808 00:40:09.976001 28084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:09.975783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:58035)
0808 00:40:09.975910 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:09.975956 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260808 00:40:10.208014 28087 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:10.208119 28087 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:10.208297 28087 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
W20260808 00:40:10.208657 27380 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
W20260808 00:40:10.208729 27385 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:10.208822 27380 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae824ee7574546c1823995d6be3596ce; no voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff
I20260808 00:40:10.208911 28087 raft_consensus.cc:2726] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:40:10.768338 28088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:10.768133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:45785)
0808 00:40:10.768256 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:10.768288 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260808 00:40:10.856168 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:10.855978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:51517)
0808 00:40:10.856090 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:10.856124 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
I20260808 00:40:10.976612 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:10.976379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:47959)
0808 00:40:10.976529 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:10.976567 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260808 00:40:11.768944 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:11.768730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:38373)
0808 00:40:11.768856 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:11.768888 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260808 00:40:11.856786 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:11.856548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:43823)
0808 00:40:11.856684 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:11.856727 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260808 00:40:11.977344 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:11.977086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:55287)
0808 00:40:11.977237 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:11.977269 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260808 00:40:12.769639 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:12.769431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:53371)
0808 00:40:12.769554 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:12.769590 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260808 00:40:12.857458 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:12.857232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:34747)
0808 00:40:12.857369 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:12.857412 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260808 00:40:12.977958 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:12.977729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:58105)
0808 00:40:12.977871 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:12.977912 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260808 00:40:13.499464 28104 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:13.499569 28104 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:13.499742 28104 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
W20260808 00:40:13.500164 27385 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
W20260808 00:40:13.500268 27380 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:13.500350 27380 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae824ee7574546c1823995d6be3596ce; no voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff
I20260808 00:40:13.500427 28104 raft_consensus.cc:2726] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:40:13.770220 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:13.770010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:49279)
0808 00:40:13.770139 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:13.770171 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260808 00:40:13.858126 28109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:13.857885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:57287)
0808 00:40:13.858018 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:13.858055 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":69,"threads_started":1}
I20260808 00:40:13.978559 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:13.978341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:45835)
0808 00:40:13.978459 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:13.978494 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":63,"threads_started":1}
I20260808 00:40:14.770834 28113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:14.770618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:44541)
0808 00:40:14.770750 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:14.770786 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260808 00:40:14.858758 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:14.858548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:34881)
0808 00:40:14.858678 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:14.858711 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260808 00:40:14.979205 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:14.978976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:51881)
0808 00:40:14.979120 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:14.979156 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260808 00:40:15.771535 28119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:15.771283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:50159)
0808 00:40:15.771450 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:15.771487 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":104,"threads_started":1}
I20260808 00:40:15.859364 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:15.859145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:56787)
0808 00:40:15.859280 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:15.859315 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
I20260808 00:40:15.979825 28121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:15.979612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:34789)
0808 00:40:15.979746 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:15.979781 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260808 00:40:16.772179 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:16.771979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:53635)
0808 00:40:16.772098 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:16.772129 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":65,"threads_started":1}
I20260808 00:40:16.860080 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:16.859851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:47983)
0808 00:40:16.859989 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:16.860028 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260808 00:40:16.980450 28125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:16.980241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:47445)
0808 00:40:16.980365 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:16.980397 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
I20260808 00:40:17.772888 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:17.772656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:43279)
0808 00:40:17.772803 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:17.772838 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260808 00:40:17.860731 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:17.860505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:41701)
0808 00:40:17.860644 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:17.860680 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260808 00:40:17.981129 28128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:17.980911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:33771)
0808 00:40:17.981037 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:17.981078 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
I20260808 00:40:18.773623 28131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:18.773408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:44053)
0808 00:40:18.773537 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:18.773571 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":70,"threads_started":1}
I20260808 00:40:18.861441 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:18.861185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:57235)
0808 00:40:18.861338 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:18.861379 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":67,"threads_started":1}
I20260808 00:40:18.981787 28133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:18.981587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:56851)
0808 00:40:18.981706 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:18.981739 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260808 00:40:19.321374 28135 raft_consensus.cc:482] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:19.321476 28135 raft_consensus.cc:504] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c6bb9ef78014f03a7235db8d743b3ff" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 38449 } } peers { permanent_uuid: "571f38c1f2054a5ea94836b0e100e523" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 34559 } } peers { permanent_uuid: "ae824ee7574546c1823995d6be3596ce" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44009 } }
I20260808 00:40:19.321667 28135 leader_election.cc:288] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449), 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559)
W20260808 00:40:19.322011 27385 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c6bb9ef78014f03a7235db8d743b3ff (127.1.208.3:38449): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
W20260808 00:40:19.322085 27380 leader_election.cc:341] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 571f38c1f2054a5ea94836b0e100e523 (127.1.208.2:34559): Not found: Tablet not found: 7264cac913c0431f8f338cdd6ada3d37
I20260808 00:40:19.322126 27380 leader_election.cc:302] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ae824ee7574546c1823995d6be3596ce; no voters: 571f38c1f2054a5ea94836b0e100e523, 8c6bb9ef78014f03a7235db8d743b3ff
I20260808 00:40:19.322186 28135 raft_consensus.cc:2726] T 7264cac913c0431f8f338cdd6ada3d37 P ae824ee7574546c1823995d6be3596ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:40:19.774330 28136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:19.774114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.1:48195)
0808 00:40:19.774249 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:19.774282 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260808 00:40:19.862113 28138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:19.861915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.2:55161)
0808 00:40:19.862031 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:19.862066 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260808 00:40:19.982471 28140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:19.982270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:38045 (local address 127.1.208.3:44969)
0808 00:40:19.982383 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:19.982425 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:38045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260808 00:40:19.988582 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 27311
I20260808 00:40:20.005617 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 27584
I20260808 00:40:20.019768 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 27813
2026-08-08T00:40:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18342 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:40:20 dist-test-slave-gjw7 krb5kdc[28148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:40:20 dist-test-slave-gjw7 krb5kdc[28148](info): set up 2 sockets
Aug 08 00:40:20 dist-test-slave-gjw7 krb5kdc[28148](info): commencing operation
krb5kdc: starting...
W20260808 00:40:21.204710 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:40:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:40:21.326164 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:21.411706 28168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:21.411928 28168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:21.411959 28168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:21.413092 28168 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:40:21.413131 28168 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:40:21.413147 28168 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:40:21.413161 28168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:21.413178 28168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:21.413210 28168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:21.413237 28168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:21.413260 28168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:21.414808 28168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:21.415063 28168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:21.415277 28168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:21.420452 28176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:21.420445 28178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.420501 28168 server_base.cc:1034] running on GCE node
W20260808 00:40:21.420467 28183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.420917 28168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:21.421172 28168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:21.422295 28168 hybrid_clock.cc:648] HybridClock initialized: now 1786149621422279 us; error 32 us; skew 500 ppm
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:21.424662 28168 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:21.425088 28168 webserver.cc:460] Webserver started at http://127.1.208.62:43703/ using document root <none> and password file <none>
I20260808 00:40:21.425324 28168 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:21.425374 28168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:21.425524 28168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:21.426553 28168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "02969e29d1a4498e8caedd350797ea43"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "9e2eb81b69c2a80846a072e64f9a448c"
server_key_iv: "5d0984aab9b299fe2d5c29f79661ca96"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.426884 28168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "02969e29d1a4498e8caedd350797ea43"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "9e2eb81b69c2a80846a072e64f9a448c"
server_key_iv: "5d0984aab9b299fe2d5c29f79661ca96"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.428160 28168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:40:21.428807 28195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.428975 28168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:21.429054 28168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "02969e29d1a4498e8caedd350797ea43"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "9e2eb81b69c2a80846a072e64f9a448c"
server_key_iv: "5d0984aab9b299fe2d5c29f79661ca96"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.429121 28168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:21.443912 28168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:21.445744 28168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:21.445863 28168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:21.449865 28168 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34455
I20260808 00:40:21.449898 28278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34455 every 8 connection(s)
I20260808 00:40:21.450212 28168 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:21.450558 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 28168
I20260808 00:40:21.450657 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:40:21.450857 1856 external_mini_cluster.cc:1428] Setting key b404923143e882226c8a58cc65b06ea6
I20260808 00:40:21.451154 28279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:21.454444 28279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap starting.
I20260808 00:40:21.455054 28279 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:21.455379 28279 log.cc:824] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:21.456059 28279 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: No bootstrap required, opened a new log
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:21.457955 28279 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:21.458132 28279 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:21.458166 28279 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Initialized, Role: FOLLOWER
I20260808 00:40:21.458284 28279 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:21.458364 28279 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:21.458422 28279 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:21.458474 28279 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:21.459128 28279 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:21.459241 28279 leader_election.cc:302] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02969e29d1a4498e8caedd350797ea43; no voters:
I20260808 00:40:21.459419 28279 leader_election.cc:288] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:40:21.459450 28292 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:21.459578 28292 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 LEADER]: Becoming Leader. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Running, Role: LEADER
I20260808 00:40:21.459681 28279 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:21.459667 28292 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:21.460182 28294 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02969e29d1a4498e8caedd350797ea43. Latest consensus state: current_term: 1 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:40:21.460186 28293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:40:21.460290 28294 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:21.460302 28293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:21.460953 28305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:40:21.461571 28305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:21.461786 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.451332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.1.208.62:34455)
0808 00:40:21.451567 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.451575 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.451608 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:40:21.452284 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.452293 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.452359 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.452502 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:21.453058 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.454378 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.455111 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.455668 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.455766 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.457935 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:21.457939 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:21.457939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:21.457949 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:21.459822 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.460399 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.460404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.460405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.460508 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.460764 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.460766 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.460766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.461106 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:21.461264 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.461419 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.461561 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":119,"threads_started":1}
I20260808 00:40:21.463020 28305 catalog_manager.cc:1347] Generated new cluster ID: af4013a510ea4f65b977d14f08e2b2f7
I20260808 00:40:21.463073 28305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:40:21.473513 28305 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:40:21.474216 28305 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:40:21.479200 28305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Generated new TSK 0
I20260808 00:40:21.479377 28305 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:40:21.504954 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:21.585753 28329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:21.585943 28329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:21.585964 28329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:21.587060 28329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:21.587100 28329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:21.587127 28329 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:40:21.588519 28329 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28329
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:21.588733 28329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:21.588938 28329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:21.593676 28344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:21.593652 28339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.593850 28329 server_base.cc:1034] running on GCE node
W20260808 00:40:21.593696 28348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.594111 28329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:21.594348 28329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:21.595484 28329 hybrid_clock.cc:648] HybridClock initialized: now 1786149621595470 us; error 36 us; skew 500 ppm
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:21.597978 28329 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:40:21.598368 28329 webserver.cc:460] Webserver started at http://127.1.208.1:35041/ using document root <none> and password file <none>
I20260808 00:40:21.598556 28329 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:21.598603 28329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:21.598724 28329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:21.599742 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "ac5dd98d1cf544629d330e75d9b2671e"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "3daca426d662735ab4e1efeb18296ed0"
server_key_iv: "4fe8cdd56fcb57c099c9ec2d997a8dee"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.600080 28329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "ac5dd98d1cf544629d330e75d9b2671e"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "3daca426d662735ab4e1efeb18296ed0"
server_key_iv: "4fe8cdd56fcb57c099c9ec2d997a8dee"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.601415 28329 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:40:21.602128 28364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.602277 28329 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:21.602375 28329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ac5dd98d1cf544629d330e75d9b2671e"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "3daca426d662735ab4e1efeb18296ed0"
server_key_iv: "4fe8cdd56fcb57c099c9ec2d997a8dee"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.602442 28329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:21.615757 28329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:21.617524 28329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:21.617651 28329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:21.617859 28329 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:21.618134 28329 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:21.618166 28329 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.618194 28329 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:21.618211 28329 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.623806 28329 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:40203
I20260808 00:40:21.623863 28558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:40203 every 8 connection(s)
I20260808 00:40:21.624138 28329 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:40:21.629837 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 28329
I20260808 00:40:21.629941 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:21.630158 1856 external_mini_cluster.cc:1428] Setting key 17868e0cfc4859709ecbc5c1320344fa
I20260808 00:40:21.633399 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.624788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:49839 (local address 127.1.208.62:34455)
0808 00:40:21.624857 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.624859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.625036 (+ 177us) server_negotiation.cc:407] Connection header received
0808 00:40:21.625609 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.625611 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.625623 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.625674 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:21.626580 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.627088 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.628035 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.628249 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.628303 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.630552 (+ 2249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:21.630554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:21.630554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:21.630562 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:21.631961 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.632360 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.632361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.632362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.632429 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.632670 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.632679 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.632679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.632820 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:21.632892 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.633172 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.633267 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:40:21.633567 28563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.624773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:49839)
0808 00:40:21.624926 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.624962 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.625513 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.625739 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.625742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.626291 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:21.626484 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.626489 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.627169 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.627170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.627954 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.627956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.628319 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.628320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.628437 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.628869 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:21.628877 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:21.630455 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:21.632033 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.632041 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.632044 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.632283 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.632491 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.632492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.632492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.632609 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.632885 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:21.632893 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:21.633047 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.633142 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.633342 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":76,"threads_started":1}
I20260808 00:40:21.634110 28561 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:21.634200 28561 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:21.634387 28561 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:21.634933 28221 ts_manager.cc:194] Registered new tserver with Master: ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203)
I20260808 00:40:21.636003 28221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:49839
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:40:21.655951 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:21.737785 28568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:21.737979 28568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:21.738001 28568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:21.739095 28568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:21.739136 28568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:21.739166 28568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:40:21.740566 28568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:21.740779 28568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:21.740983 28568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:21.745674 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:21.745718 28584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:21.745719 28576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.746016 28568 server_base.cc:1034] running on GCE node
I20260808 00:40:21.746189 28568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:21.746373 28568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:21.747510 28568 hybrid_clock.cc:648] HybridClock initialized: now 1786149621747499 us; error 28 us; skew 500 ppm
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:21.749924 28568 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:40:21.750326 28568 webserver.cc:460] Webserver started at http://127.1.208.2:33151/ using document root <none> and password file <none>
I20260808 00:40:21.750517 28568 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:21.750566 28568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:21.750687 28568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:21.751658 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "06c22f4134f20e7d1682e7eea5c299f5"
server_key_iv: "c6602ee494ab67b980886536e567d818"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.751984 28568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "06c22f4134f20e7d1682e7eea5c299f5"
server_key_iv: "c6602ee494ab67b980886536e567d818"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.753219 28568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:40:21.753924 28592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.754067 28568 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:21.754148 28568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "06c22f4134f20e7d1682e7eea5c299f5"
server_key_iv: "c6602ee494ab67b980886536e567d818"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.754209 28568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:21.771587 28568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:21.773509 28568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:21.773643 28568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:21.773859 28568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:21.774142 28568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:21.774174 28568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.774204 28568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:21.774226 28568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.779879 28568 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:38763
I20260808 00:40:21.779924 28787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:38763 every 8 connection(s)
I20260808 00:40:21.780208 28568 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:40:21.780690 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 28568
I20260808 00:40:21.780782 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:40:21.780985 1856 external_mini_cluster.cc:1428] Setting key 2ce8056b1ed824573ca8cdc48fe8b3df
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:21.790086 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.780858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:45191 (local address 127.1.208.62:34455)
0808 00:40:21.780937 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.780939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.781154 (+ 215us) server_negotiation.cc:407] Connection header received
0808 00:40:21.781793 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.781794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.781807 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.781888 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:21.782777 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.783445 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.784803 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.784992 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.785038 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.787446 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:21.787447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:21.787448 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:21.787454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:21.788681 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.789165 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.789166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.789166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.789213 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.789484 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.789485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.789485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.789585 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:21.789633 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.789960 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.790020 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:40:21.790370 28792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.780843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:45191)
0808 00:40:21.781028 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.781069 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.781711 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.781947 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.781950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.782510 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:21.782690 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.782695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.783552 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.783554 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.784727 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.784732 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.785069 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.785070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.785213 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.785677 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:21.785687 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:21.787353 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:21.788761 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.788769 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.788771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.789083 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.789269 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.789270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.789271 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.789425 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.789643 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:21.789649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:21.789815 (+ 166us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.789946 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.790117 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
I20260808 00:40:21.790825 28789 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:21.790916 28789 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:21.791086 28789 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:21.791471 28221 ts_manager.cc:194] Registered new tserver with Master: 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:21.791882 28221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:45191
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:40:21.806149 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:21.889118 28797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:21.889338 28797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:21.889371 28797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:21.890506 28797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:21.890543 28797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:21.890576 28797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:40:21.891986 28797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:21.892221 28797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:21.892436 28797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:21.897207 28804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:21.897222 28808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.897329 28797 server_base.cc:1034] running on GCE node
W20260808 00:40:21.897260 28811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:21.897601 28797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:21.897814 28797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:21.898945 28797 hybrid_clock.cc:648] HybridClock initialized: now 1786149621898934 us; error 28 us; skew 500 ppm
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:21.901346 28797 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:40:21.901782 28797 webserver.cc:460] Webserver started at http://127.1.208.3:38179/ using document root <none> and password file <none>
I20260808 00:40:21.902007 28797 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:21.902055 28797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:21.902180 28797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:40:21.903160 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "c6a64ab9a55b45e1a7a47365aba77701"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "315ce6a5d45e196928025afb21314e12"
server_key_iv: "659be3d7c7a129a52f4c300c118f904f"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.903482 28797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "c6a64ab9a55b45e1a7a47365aba77701"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "315ce6a5d45e196928025afb21314e12"
server_key_iv: "659be3d7c7a129a52f4c300c118f904f"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.904695 28797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:40:21.905457 28823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.905596 28797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.905680 28797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c6a64ab9a55b45e1a7a47365aba77701"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "315ce6a5d45e196928025afb21314e12"
server_key_iv: "659be3d7c7a129a52f4c300c118f904f"
server_key_version: "encryptionkey@0"
I20260808 00:40:21.905748 28797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:21.927093 28797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:21.928789 28797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:21.928931 28797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:21.929133 28797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:21.929476 28797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:21.929510 28797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.929539 28797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:21.929556 28797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:21.935412 28797 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:36375
I20260808 00:40:21.935483 29040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:36375 every 8 connection(s)
I20260808 00:40:21.935760 28797 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:40:21.940866 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 28797
I20260808 00:40:21.940958 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:40:21.941135 1856 external_mini_cluster.cc:1428] Setting key 1b76cc8ffe743343022870d10b1b6438
Aug 08 00:40:21 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149621, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:21.944041 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.936413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:60893 (local address 127.1.208.62:34455)
0808 00:40:21.936475 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.936477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.936643 (+ 166us) server_negotiation.cc:407] Connection header received
0808 00:40:21.937100 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.937101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.937110 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.937159 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:21.938120 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.938606 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.939516 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.939688 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.939730 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.941807 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:21.941808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:21.941809 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:21.941817 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:21.942998 (+ 1181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.943337 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.943338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.943338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.943384 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.943555 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.943555 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.943555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.943651 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:21.943693 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.943916 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.943975 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:40:21.944205 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.936371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:60893)
0808 00:40:21.936526 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.936566 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.937025 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.937218 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.937221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.937812 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:21.938042 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.938049 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.938668 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.938669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.939457 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.939459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.939739 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.939739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.939836 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.940226 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:21.940234 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:21.941720 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:21.943066 (+ 1346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.943073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.943075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.943282 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.943426 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:21.943427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:21.943427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:21.943514 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:21.943694 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:21.943697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:21.943806 (+ 109us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.943891 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.944006 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":83,"threads_started":1}
I20260808 00:40:21.944624 29041 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:21.944705 29041 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:21.944885 29041 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:21.945341 28221 ts_manager.cc:194] Registered new tserver with Master: c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375)
I20260808 00:40:21.945706 28221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:60893
I20260808 00:40:21.952751 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:40:21.959384 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.954095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.1.208.62:34455)
0808 00:40:21.954147 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.954148 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.954234 (+ 86us) server_negotiation.cc:407] Connection header received
0808 00:40:21.954274 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.954274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.954282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.954325 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:21.954822 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.955281 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.955972 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.956143 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.956183 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.957207 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:21.957209 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:21.957209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:21.957215 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:21.958537 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.958806 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.958806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.958807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.958864 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:21.958996 (+ 132us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:21.958997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:21.958997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:21.959121 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:21.959193 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.959266 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.959321 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:40:21.960811 28221 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:40:21.962536 28221 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:40:21.968278 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53498 (local address 127.1.208.1:40203)
0808 00:40:21.964853 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.964857 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.964869 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:40:21.964910 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.964912 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.964928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.965006 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:21.965167 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966017 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.967146 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.967869 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.967910 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.967969 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.968168 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.968210 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
I20260808 00:40:21.968269 29057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40203 (local address 127.0.0.1:53498)
0808 00:40:21.964691 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.964736 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.964803 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.965016 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.965019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.965022 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:21.965090 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.965092 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966104 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.966105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.967043 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.967046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.968017 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.968018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.968075 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.968084 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.968134 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.968186 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"spinlock_wait_cycles":896,"thread_start_us":60,"threads_started":1}
I20260808 00:40:21.968760 29056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38763 (local address 127.0.0.1:54708)
0808 00:40:21.964936 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.964943 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.964991 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.965389 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.965390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.965392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:21.965454 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.965456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966737 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.966738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.967636 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.967639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.968512 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.968513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.968587 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.968591 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.968650 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.968692 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":31,"thread_start_us":72,"threads_started":1}
I20260808 00:40:21.968801 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.1.208.2:38763)
0808 00:40:21.964887 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.964891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.964986 (+ 95us) server_negotiation.cc:407] Connection header received
0808 00:40:21.965054 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.965056 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.965073 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.965257 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:21.965529 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966623 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.967713 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.968339 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.968419 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.968474 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.968674 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.968723 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":90,"threads_started":1}
I20260808 00:40:21.968966 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36375 (local address 127.0.0.1:50226)
0808 00:40:21.964810 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:21.964820 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:21.964872 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:21.965378 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:21.965379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:21.965382 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:21.965443 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.965444 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966730 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.966731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.967838 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:21.967841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.968743 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.968744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:21.968810 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.968813 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:40:21.968864 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:40:21.968899 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260808 00:40:21.969274 28444 tablet_service.cc:1467] Processing CreateTablet for tablet 99f3a89213b543c2a049e1b66f90349a (DEFAULT_TABLE table=test-table [id=0e79825ac48e4f01952475766d45647d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:21.969689 28444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f3a89213b543c2a049e1b66f90349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:21.969627 28672 tablet_service.cc:1467] Processing CreateTablet for tablet 99f3a89213b543c2a049e1b66f90349a (DEFAULT_TABLE table=test-table [id=0e79825ac48e4f01952475766d45647d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:21.969901 28672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f3a89213b543c2a049e1b66f90349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:21.972257 29063 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap starting.
I20260808 00:40:21.972465 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:21.964632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50226 (local address 127.1.208.3:36375)
0808 00:40:21.965132 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:21.965135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:21.965144 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:40:21.965178 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:21.965180 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:21.965193 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:21.965272 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:21.965524 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.966635 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.967932 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:21.968605 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:21.968661 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:21.968712 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:21.969123 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 00:40:21.972384 (+ 3261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":89,"threads_started":1}
I20260808 00:40:21.972617 29064 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Bootstrap starting.
I20260808 00:40:21.972906 29063 tablet_bootstrap.cc:654] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:21.973254 29064 tablet_bootstrap.cc:654] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:21.973176 28919 tablet_service.cc:1467] Processing CreateTablet for tablet 99f3a89213b543c2a049e1b66f90349a (DEFAULT_TABLE table=test-table [id=0e79825ac48e4f01952475766d45647d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:21.973413 29063 log.cc:824] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:21.973548 28919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 99f3a89213b543c2a049e1b66f90349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:21.973840 29064 log.cc:824] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:21.974139 29063 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: No bootstrap required, opened a new log
I20260808 00:40:21.974215 29063 ts_tablet_manager.cc:1397] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:40:21.974680 29064 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: No bootstrap required, opened a new log
I20260808 00:40:21.974753 29064 ts_tablet_manager.cc:1397] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:40:21.975649 29063 raft_consensus.cc:348] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.975787 29063 raft_consensus.cc:374] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:21.975818 29063 raft_consensus.cc:729] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5dd98d1cf544629d330e75d9b2671e, State: Initialized, Role: FOLLOWER
I20260808 00:40:21.975965 29063 consensus_queue.cc:260] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.976229 29063 ts_tablet_manager.cc:1428] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:40:21.976259 28561 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
I20260808 00:40:21.976694 29072 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Bootstrap starting.
I20260808 00:40:21.976676 29064 raft_consensus.cc:348] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.976851 29064 raft_consensus.cc:374] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:21.976892 29064 raft_consensus.cc:729] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a8d89f25494929a7f25d43e8f2e2ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:21.977000 29064 consensus_queue.cc:260] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.977283 28789 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
I20260808 00:40:21.977267 29064 ts_tablet_manager.cc:1428] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:40:21.977437 29072 tablet_bootstrap.cc:654] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:21.977967 29072 log.cc:824] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:21.978587 29072 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: No bootstrap required, opened a new log
I20260808 00:40:21.978641 29072 ts_tablet_manager.cc:1397] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:40:21.979882 29072 raft_consensus.cc:348] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.980014 29072 raft_consensus.cc:374] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:21.980036 29072 raft_consensus.cc:729] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a64ab9a55b45e1a7a47365aba77701, State: Initialized, Role: FOLLOWER
I20260808 00:40:21.980109 29072 consensus_queue.cc:260] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:21.980297 29072 ts_tablet_manager.cc:1428] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:40:21.980338 29041 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
W20260808 00:40:22.030786 28790 tablet.cc:2369] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:40:22.071648 29076 raft_consensus.cc:482] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:22.071810 29076 raft_consensus.cc:504] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:22.072206 29076 leader_election.cc:288] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:22.075826 29060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.072446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:51989 (local address 127.1.208.1:40203)
0808 00:40:22.072496 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:22.072498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:22.072526 (+ 28us) server_negotiation.cc:407] Connection header received
0808 00:40:22.072559 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:22.072561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:22.072573 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:22.072626 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:22.072780 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.073559 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.074899 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.075479 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.075549 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:22.075560 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:22.075710 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 00:40:22.075749 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:40:22.075842 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.072694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:41227 (local address 127.1.208.2:38763)
0808 00:40:22.072751 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:22.072753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:22.072762 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:40:22.072788 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:22.072788 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:22.072798 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:22.072855 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:22.072963 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.073653 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.074899 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.075479 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.075549 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:22.075560 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:22.075710 (+ 150us) server_negotiation.cc:299] Negotiation successful
0808 00:40:22.075749 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:40:22.075884 29077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.072387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38763 (local address 127.1.208.3:41227)
0808 00:40:22.072658 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:22.072667 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:22.072710 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:22.072858 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:22.072858 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:22.072859 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:22.072910 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:22.072911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.073721 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.073721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:22.074826 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:22.074828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.075554 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.075555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:22.075642 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:22.075647 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:40:22.075700 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:40:22.075748 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"mutex_wait_us":94,"thread_start_us":108,"threads_started":1}
I20260808 00:40:22.075856 29043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.072381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40203 (local address 127.1.208.3:51989)
0808 00:40:22.072428 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:22.072440 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:22.072486 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:22.072639 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:22.072640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:22.072641 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:22.072718 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:22.072720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.073629 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.073629 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:22.074812 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:22.074816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.075554 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.075555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:22.075645 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:22.075647 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:40:22.075700 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:40:22.075748 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19,"mutex_wait_us":4}
I20260808 00:40:22.076346 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f3a89213b543c2a049e1b66f90349a" candidate_uuid: "c6a64ab9a55b45e1a7a47365aba77701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac5dd98d1cf544629d330e75d9b2671e" is_pre_election: true
I20260808 00:40:22.076514 28480 raft_consensus.cc:2454] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a64ab9a55b45e1a7a47365aba77701 in term 0.
I20260808 00:40:22.076537 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f3a89213b543c2a049e1b66f90349a" candidate_uuid: "c6a64ab9a55b45e1a7a47365aba77701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" is_pre_election: true
I20260808 00:40:22.076695 28714 raft_consensus.cc:2454] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6a64ab9a55b45e1a7a47365aba77701 in term 0.
I20260808 00:40:22.076771 28826 leader_election.cc:302] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e, c6a64ab9a55b45e1a7a47365aba77701; no voters:
I20260808 00:40:22.076903 29076 raft_consensus.cc:2781] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:40:22.076953 29076 raft_consensus.cc:482] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:40:22.076970 29076 raft_consensus.cc:3037] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:22.077694 29076 raft_consensus.cc:504] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:22.077817 29076 leader_election.cc:288] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [CANDIDATE]: Term 1 election: Requested vote from peers ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:22.077962 28714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f3a89213b543c2a049e1b66f90349a" candidate_uuid: "c6a64ab9a55b45e1a7a47365aba77701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
I20260808 00:40:22.077972 28480 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "99f3a89213b543c2a049e1b66f90349a" candidate_uuid: "c6a64ab9a55b45e1a7a47365aba77701" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac5dd98d1cf544629d330e75d9b2671e"
I20260808 00:40:22.078042 28714 raft_consensus.cc:3037] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:22.078047 28480 raft_consensus.cc:3037] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:22.078730 28480 raft_consensus.cc:2454] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a64ab9a55b45e1a7a47365aba77701 in term 1.
I20260808 00:40:22.078794 28714 raft_consensus.cc:2454] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6a64ab9a55b45e1a7a47365aba77701 in term 1.
I20260808 00:40:22.078890 28826 leader_election.cc:302] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e, c6a64ab9a55b45e1a7a47365aba77701; no voters:
I20260808 00:40:22.078974 29076 raft_consensus.cc:2781] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:22.079130 29076 raft_consensus.cc:686] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 1 LEADER]: Becoming Leader. State: Replica: c6a64ab9a55b45e1a7a47365aba77701, State: Running, Role: LEADER
I20260808 00:40:22.079224 29076 consensus_queue.cc:237] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:22.080097 28221 catalog_manager.cc:5495] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 reported cstate change: term changed from 0 to 1, leader changed from <none> to c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3). New cstate: current_term: 1 leader_uuid: "c6a64ab9a55b45e1a7a47365aba77701" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } health_report { overall_health: UNKNOWN } } }
I20260808 00:40:22.112494 29059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.110239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50236 (local address 127.1.208.3:36375)
0808 00:40:22.110307 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:22.110309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:22.110317 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:40:22.110355 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:22.110356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:22.110377 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:22.110443 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:22.110577 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.111033 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.111761 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:22.111964 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:22.112017 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:22.112173 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:22.112276 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:40:22.112346 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:22.112386 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:40:22.112414 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:40:22.114365 28480 raft_consensus.cc:1261] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Refusing update from remote peer c6a64ab9a55b45e1a7a47365aba77701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:22.114440 28714 raft_consensus.cc:1261] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Refusing update from remote peer c6a64ab9a55b45e1a7a47365aba77701: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:22.114725 29078 consensus_queue.cc:1035] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:22.114879 29076 consensus_queue.cc:1035] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:22.117704 29085 mvcc.cc:205] Tried to move back new op lower bound from 7316068852177780736 to 7316068852037177344. Current Snapshot: MvccSnapshot[applied={T|T < 7316068852177780736}]
I20260808 00:40:22.118010 29086 mvcc.cc:205] Tried to move back new op lower bound from 7316068852177780736 to 7316068852037177344. Current Snapshot: MvccSnapshot[applied={T|T < 7316068852177780736}]
I20260808 00:40:22.119369 28220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:22.119465 28220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:22.120231 28220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Sending DeleteTablet for 3 replicas of tablet 99f3a89213b543c2a049e1b66f90349a
I20260808 00:40:22.120582 28672 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f3a89213b543c2a049e1b66f90349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:22 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:54708
I20260808 00:40:22.120590 28444 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f3a89213b543c2a049e1b66f90349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:22 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:53498
I20260808 00:40:22.120834 29108 tablet_replica.cc:330] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: stopping tablet replica
I20260808 00:40:22.120827 28919 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f3a89213b543c2a049e1b66f90349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:22 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:50226
I20260808 00:40:22.120931 29108 raft_consensus.cc:2229] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:22.121029 29108 raft_consensus.cc:2258] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:22.121049 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 28329
I20260808 00:40:22.130165 29107 tablet_replica.cc:330] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: stopping tablet replica
I20260808 00:40:22.130342 29107 raft_consensus.cc:2229] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:22.130512 29107 raft_consensus.cc:2258] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:22.130749 29109 tablet_replica.cc:330] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: stopping tablet replica
I20260808 00:40:22.130892 29109 raft_consensus.cc:2229] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:40:22.131038 29109 raft_consensus.cc:2258] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:22.131141 29107 ts_tablet_manager.cc:1905] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:22.131626 29109 ts_tablet_manager.cc:1905] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:22.133090 29107 ts_tablet_manager.cc:1918] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:22.133181 29107 log.cc:1196] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/99f3a89213b543c2a049e1b66f90349a
I20260808 00:40:22.133487 29109 ts_tablet_manager.cc:1918] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:22.133556 29107 ts_tablet_manager.cc:1939] T 99f3a89213b543c2a049e1b66f90349a P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting consensus metadata
I20260808 00:40:22.133656 29109 log.cc:1196] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/99f3a89213b543c2a049e1b66f90349a
I20260808 00:40:22.133960 29109 ts_tablet_manager.cc:1939] T 99f3a89213b543c2a049e1b66f90349a P c6a64ab9a55b45e1a7a47365aba77701: Deleting consensus metadata
I20260808 00:40:22.134058 28196 catalog_manager.cc:4841] TS 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): tablet 99f3a89213b543c2a049e1b66f90349a (table test-table [id=0e79825ac48e4f01952475766d45647d]) successfully deleted
I20260808 00:40:22.134414 28200 catalog_manager.cc:4841] TS c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): tablet 99f3a89213b543c2a049e1b66f90349a (table test-table [id=0e79825ac48e4f01952475766d45647d]) successfully deleted
W20260808 00:40:22.135300 28199 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:40203 (error 108)
I20260808 00:40:22.135515 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 28568
I20260808 00:40:22.142755 29053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:22.135657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40203 (local address 127.0.0.1:53510)
0808 00:40:22.142653 (+ 6996us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:22.142714 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6966}
W20260808 00:40:22.145457 28199 catalog_manager.cc:4584] TS ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 99f3a89213b543c2a049e1b66f90349a: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40203: connect: Connection refused (error 111)
I20260808 00:40:22.148128 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 28797
I20260808 00:40:22.163009 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 28168
I20260808 00:40:42.172174 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--webserver_interface=127.1.208.62
--webserver_port=43703
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:40:42.259308 29112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:42.259526 29112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:42.259548 29112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:42.260689 29112 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:40:42.260735 29112 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:40:42.260748 29112 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:40:42.260774 29112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:40:42.260803 29112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:42.260831 29112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:42.260859 29112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:40:42.260887 29112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:40:42.262435 29112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=43703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:42.262697 29112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:42.262928 29112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:42.272418 29127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.272526 29112 server_base.cc:1034] running on GCE node
W20260808 00:40:42.272418 29122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.272452 29120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.272934 29112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:42.273253 29112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:42.274442 29112 hybrid_clock.cc:648] HybridClock initialized: now 1786149642274420 us; error 38 us; skew 500 ppm
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:42.280218 29112 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:40:42.280648 29112 webserver.cc:460] Webserver started at http://127.1.208.62:43703/ using document root <none> and password file <none>
I20260808 00:40:42.280860 29112 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:42.280933 29112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:42.282307 29112 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:42.282923 29142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.283074 29112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:42.283164 29112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "02969e29d1a4498e8caedd350797ea43"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "9e2eb81b69c2a80846a072e64f9a448c"
server_key_iv: "5d0984aab9b299fe2d5c29f79661ca96"
server_key_version: "encryptionkey@0"
I20260808 00:40:42.283584 29112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:42.297652 29112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:42.299444 29112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:42.299578 29112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:42.303673 29112 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34455
I20260808 00:40:42.303706 29234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34455 every 8 connection(s)
I20260808 00:40:42.304085 29112 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:40:42.305634 29235 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:40:42.306260 29235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap starting.
I20260808 00:40:42.309008 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 29112
I20260808 00:40:42.309265 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:40203
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=35041
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:42.314942 29235 log.cc:824] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:42.316757 29235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:42.317041 29235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap complete.
I20260808 00:40:42.319200 29235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:42.319432 29235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Initialized, Role: FOLLOWER
I20260808 00:40:42.319574 29235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:42.319636 29235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:40:42.319680 29235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:40:42.319728 29235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 1 FOLLOWER]: Advancing to term 2
I20260808 00:40:42.320704 29235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:42.320794 29235 leader_election.cc:302] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02969e29d1a4498e8caedd350797ea43; no voters:
I20260808 00:40:42.320946 29235 leader_election.cc:288] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 00:40:42.320995 29242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 00:40:42.321128 29242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 LEADER]: Becoming Leader. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Running, Role: LEADER
I20260808 00:40:42.321215 29235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:40:42.321213 29242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:40:42.321815 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:40:42.321915 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:42.322471 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02969e29d1a4498e8caedd350797ea43. Latest consensus state: current_term: 2 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:40:42.322551 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:40:42.322757 29252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:40:42.323556 29252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:40:42.323648 29252 catalog_manager.cc:1259] Loaded cluster ID: af4013a510ea4f65b977d14f08e2b2f7
I20260808 00:40:42.323680 29252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:40:42.324452 29252 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:40:42.324615 29252 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Loaded TSK: 0
I20260808 00:40:42.331532 29252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Generated new TSK 1
I20260808 00:40:42.331621 29252 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:40:42.404810 29237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:42.405014 29237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:42.405045 29237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:42.406181 29237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:42.406229 29237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:42.406265 29237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:40:42.407680 29237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:40203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=35041
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:42.407900 29237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:42.408109 29237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:42.413975 29293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.413959 29294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.414328 29237 server_base.cc:1034] running on GCE node
W20260808 00:40:42.414381 29296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.414631 29237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:42.414922 29237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:42.416103 29237 hybrid_clock.cc:648] HybridClock initialized: now 1786149642416089 us; error 27 us; skew 500 ppm
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:42.419227 29237 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:40:42.419751 29237 webserver.cc:460] Webserver started at http://127.1.208.1:35041/ using document root <none> and password file <none>
I20260808 00:40:42.420008 29237 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:42.420087 29237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:42.421891 29237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:40:42.422631 29303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.422806 29237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:42.422888 29237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ac5dd98d1cf544629d330e75d9b2671e"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "3daca426d662735ab4e1efeb18296ed0"
server_key_iv: "4fe8cdd56fcb57c099c9ec2d997a8dee"
server_key_version: "encryptionkey@0"
I20260808 00:40:42.423247 29237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:42.441036 29237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:42.443122 29237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:42.443238 29237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:42.443455 29237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:42.443881 29316 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:40:42.445740 29237 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:40:42.445783 29237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:40:42.445818 29237 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:40:42.446446 29237 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:40:42.446478 29237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:42.446553 29316 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap starting.
I20260808 00:40:42.453455 29237 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:40203
I20260808 00:40:42.453819 29237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:40:42.455096 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 29237
I20260808 00:40:42.455385 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:38763
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=33151
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:42.457500 29499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:40203 every 8 connection(s)
I20260808 00:40:42.458822 29316 log.cc:824] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:42.461315 29316 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:40:42.461613 29316 tablet_bootstrap.cc:492] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap complete.
I20260808 00:40:42.462803 29316 ts_tablet_manager.cc:1397] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:42.464422 29316 raft_consensus.cc:348] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:42.464706 29316 raft_consensus.cc:729] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5dd98d1cf544629d330e75d9b2671e, State: Initialized, Role: FOLLOWER
I20260808 00:40:42.464864 29316 consensus_queue.cc:260] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } } peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } }
I20260808 00:40:42.465126 29316 ts_tablet_manager.cc:1428] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260808 00:40:42.468864 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.454547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:36705)
0808 00:40:42.454753 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.454799 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.455397 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.458318 (+ 2921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.458324 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.458625 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:42.458821 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.458827 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.460430 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.460431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.461770 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.461773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.462614 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.462615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.462769 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.463376 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:42.463387 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:42.465133 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:42.467292 (+ 2159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.467298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.467300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.467541 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.467814 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.467815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.467815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.467906 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.468345 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:42.468349 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:42.468500 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.468599 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.468722 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":110,"threads_started":1}
I20260808 00:40:42.469022 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.454614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:36705 (local address 127.1.208.62:34455)
0808 00:40:42.457840 (+ 3226us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.457851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.457895 (+ 44us) server_negotiation.cc:407] Connection header received
0808 00:40:42.458034 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.458042 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.458101 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.458941 (+ 840us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.459030 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.460277 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.461889 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.462497 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.463072 (+ 575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.465263 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.465268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.465268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.465282 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.467204 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.467626 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.467629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.467629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.467745 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.467961 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.467962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.467963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.468266 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.468383 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.468684 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.468829 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3168,"thread_start_us":72,"threads_started":1}
I20260808 00:40:42.469641 29501 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:42.469741 29501 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:42.469918 29501 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:42.470664 29168 ts_manager.cc:194] Registered new tserver with Master: ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203)
I20260808 00:40:42.471685 29168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:36705
I20260808 00:40:42.472505 29501 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.1@KRBTEST.COM
I20260808 00:40:42.479535 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.471224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40203 (local address 127.0.0.1:33782)
0808 00:40:42.471448 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.471479 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.471549 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.471724 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.471727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.472083 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:42.472140 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.472142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.473436 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.473437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.473982 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.473984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.474562 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.474562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.474611 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.475026 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:42.475030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:42.476648 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:42.478518 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.478525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.478526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.478723 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.478916 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.478916 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.478916 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.478966 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.479252 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:42.479259 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:42.479305 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.479373 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.479467 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260808 00:40:42.479733 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.471297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.1.208.1:40203)
0808 00:40:42.471452 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.471456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.471541 (+ 85us) server_negotiation.cc:407] Connection header received
0808 00:40:42.471626 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.471628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.471641 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.471715 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.472229 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.473363 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.474050 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.474496 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.474553 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.476729 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.476738 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.476738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.476745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.478435 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.478794 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.478797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.478797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.478862 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.479012 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.479013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.479013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.479179 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.479260 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.479423 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.479519 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":89,"threads_started":1}
I20260808 00:40:42.480057 29382 tablet_service.cc:1514] Processing DeleteTablet for tablet 99f3a89213b543c2a049e1b66f90349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:22 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33782
I20260808 00:40:42.480638 29515 tablet_replica.cc:330] stopping tablet replica
I20260808 00:40:42.480754 29515 raft_consensus.cc:2229] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:42.480829 29515 raft_consensus.cc:2258] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:42.481140 29515 ts_tablet_manager.cc:1905] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:42.483177 29515 ts_tablet_manager.cc:1918] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:42.483248 29515 log.cc:1196] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal/wals/99f3a89213b543c2a049e1b66f90349a
I20260808 00:40:42.483453 29515 ts_tablet_manager.cc:1939] T 99f3a89213b543c2a049e1b66f90349a P ac5dd98d1cf544629d330e75d9b2671e: Deleting consensus metadata
I20260808 00:40:42.483932 29147 catalog_manager.cc:4841] TS ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203): tablet 99f3a89213b543c2a049e1b66f90349a (table test-table [id=0e79825ac48e4f01952475766d45647d]) successfully deleted
W20260808 00:40:42.546454 29508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:42.546648 29508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:42.546676 29508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:42.547806 29508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:42.547845 29508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:42.547879 29508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:40:42.549401 29508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:38763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=33151
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:42.549644 29508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:42.549853 29508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:42.554611 29539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.554612 29529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.554632 29535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.554636 29508 server_base.cc:1034] running on GCE node
I20260808 00:40:42.555075 29508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:42.555292 29508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:42.556440 29508 hybrid_clock.cc:648] HybridClock initialized: now 1786149642556429 us; error 29 us; skew 500 ppm
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:42.558897 29508 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:40:42.559331 29508 webserver.cc:460] Webserver started at http://127.1.208.2:33151/ using document root <none> and password file <none>
I20260808 00:40:42.559558 29508 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:42.559621 29508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:42.560894 29508 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:42.561440 29552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.561587 29508 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:42.561681 29508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "06c22f4134f20e7d1682e7eea5c299f5"
server_key_iv: "c6602ee494ab67b980886536e567d818"
server_key_version: "encryptionkey@0"
I20260808 00:40:42.562013 29508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:42.576148 29508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:42.577939 29508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:42.578069 29508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:42.578280 29508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:42.578578 29508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:42.578612 29508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.578642 29508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:42.578667 29508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.584662 29508 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:38763
I20260808 00:40:42.584717 29744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:38763 every 8 connection(s)
I20260808 00:40:42.585043 29508 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:42.592645 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 29508
I20260808 00:40:42.592923 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:36375
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=38179
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:40:42.594125 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.585818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:43335 (local address 127.1.208.62:34455)
0808 00:40:42.585880 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.585882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.586083 (+ 201us) server_negotiation.cc:407] Connection header received
0808 00:40:42.586583 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.586584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.586599 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.586657 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.587622 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.588134 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.589165 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.589442 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.589514 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.591499 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.591500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.591500 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.591508 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.592712 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.593156 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.593157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.593157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.593222 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.593485 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.593486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.593487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.593606 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.593666 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.593931 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.594031 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:40:42.594290 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.585776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:43335)
0808 00:40:42.585932 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.585975 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.586488 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.586715 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.586718 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.587303 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:42.587552 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.587558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.588218 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.588219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.589088 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.589091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.589515 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.589515 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.589623 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.590046 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:42.590054 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:42.591413 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:42.592803 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.592809 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.592811 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.593081 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.593279 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.593280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.593280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.593425 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.593663 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:42.593670 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:42.593807 (+ 137us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.593894 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.594074 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260808 00:40:42.594719 29746 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:42.594820 29746 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:42.594998 29746 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:42.595378 29168 ts_manager.cc:194] Registered new tserver with Master: 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:42.595741 29168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:43335
W20260808 00:40:42.676195 29752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:40:42.676393 29752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:40:42.676425 29752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:40:42.677573 29752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:40:42.677613 29752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:40:42.677649 29752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:40:42.679116 29752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:36375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=38179
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:40:42.679340 29752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:40:42.679553 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:40:42.684142 29763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.684173 29759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:40:42.684155 29765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:40:42.684330 29752 server_base.cc:1034] running on GCE node
I20260808 00:40:42.684509 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:40:42.684711 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:40:42.685842 29752 hybrid_clock.cc:648] HybridClock initialized: now 1786149642685820 us; error 33 us; skew 500 ppm
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:40:42.688270 29752 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:40:42.688705 29752 webserver.cc:460] Webserver started at http://127.1.208.3:38179/ using document root <none> and password file <none>
I20260808 00:40:42.688899 29752 fs_manager.cc:359] Metadata directory not provided
I20260808 00:40:42.688970 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:40:42.690280 29752 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:40:42.690793 29778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.690953 29752 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:40:42.691035 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c6a64ab9a55b45e1a7a47365aba77701"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "315ce6a5d45e196928025afb21314e12"
server_key_iv: "659be3d7c7a129a52f4c300c118f904f"
server_key_version: "encryptionkey@0"
I20260808 00:40:42.691341 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:40:42.715943 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:40:42.717703 29752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:40:42.717837 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:40:42.718053 29752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:40:42.718344 29752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:40:42.718379 29752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.718407 29752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:40:42.718425 29752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:40:42.724524 29752 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:36375
I20260808 00:40:42.724584 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:36375 every 8 connection(s)
I20260808 00:40:42.724896 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:40:42.729151 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 29752
Aug 08 00:40:42 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149642, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:40:42.734189 29506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.725663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:52301 (local address 127.1.208.62:34455)
0808 00:40:42.725716 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.725718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.725906 (+ 188us) server_negotiation.cc:407] Connection header received
0808 00:40:42.726399 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.726400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.726410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.726465 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.727379 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.727863 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.728769 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.728963 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.729028 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.731379 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.731380 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.731380 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.731387 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.732855 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.733230 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.733231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.733231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.733319 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.733554 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.733554 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.733555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.733672 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.733727 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.734029 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.734124 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:40:42.734395 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.725631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:52301)
0808 00:40:42.725787 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.725826 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.726314 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.726520 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.726523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.727106 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:40:42.727299 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.727304 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.727942 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.727942 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.728701 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.728704 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.729036 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.729036 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.729137 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.729726 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:40:42.729735 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:40:42.731300 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:40:42.732943 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.732951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.732953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.733163 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.733389 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:40:42.733389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:40:42.733390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:40:42.733492 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:40:42.733737 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:40:42.733744 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:40:42.733896 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.734009 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.734172 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
I20260808 00:40:42.734844 29987 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34455
I20260808 00:40:42.734948 29987 heartbeater.cc:464] Registering TS with master...
I20260808 00:40:42.735139 29987 heartbeater.cc:511] Master 127.1.208.62:34455 requested a full tablet report, sending...
I20260808 00:40:42.735567 29168 ts_manager.cc:194] Registered new tserver with Master: c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375)
I20260808 00:40:42.736032 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.729568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.1.208.62:34455)
0808 00:40:42.729692 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.729694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.729813 (+ 119us) server_negotiation.cc:407] Connection header received
0808 00:40:42.729882 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.729883 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.729894 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.729949 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.730610 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.731204 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.732073 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.732254 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.732300 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.733558 (+ 1258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.733559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.733559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.733566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.735035 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.735367 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.735368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.735369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.735431 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.735613 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.735613 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.735614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.735709 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.735828 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.735896 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.735970 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":67,"threads_started":1}
I20260808 00:40:42.736033 29168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:52301
I20260808 00:40:42.736768 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:40:42.739183 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.737101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56768 (local address 127.1.208.62:34455)
0808 00:40:42.737135 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.737137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.737249 (+ 112us) server_negotiation.cc:407] Connection header received
0808 00:40:42.737320 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.737321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.737361 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.737424 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:42.737574 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.737986 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.738675 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.738844 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.738884 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.738980 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:42.739016 (+ 36us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 00:40:42.739132 (+ 116us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56768: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 00:40:42.739238 29994 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56768: token expired
W20260808 00:40:42.739317 29045 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.208.62:34455)
I20260808 00:40:42.744138 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.739524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56780 (local address 127.1.208.62:34455)
0808 00:40:42.739558 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.739560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.739566 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:40:42.739618 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.739618 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.739625 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.739666 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:40:42.740037 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.740469 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.741081 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.741254 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.741360 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.742237 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:40:42.742237 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:40:42.742238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:40:42.742242 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:40:42.743430 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.743655 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.743655 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.743655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.743694 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:40:42.743818 (+ 124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:40:42.743818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:40:42.743818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:40:42.743901 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:40:42.743963 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.744033 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.744085 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:40:42.745087 29168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56780:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:40:42.746680 29168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:40:42.748632 29382 tablet_service.cc:1467] Processing CreateTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 (DEFAULT_TABLE table=test-table [id=9aca1e94392c4cdf8cf73c7d361f0192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:42.748847 29382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e264bfd7e3349f186f0b9fa807e8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:42.750502 29316 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap starting.
I20260808 00:40:42.750834 29316 tablet_bootstrap.cc:654] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:42.751380 29316 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: No bootstrap required, opened a new log
I20260808 00:40:42.751433 29316 ts_tablet_manager.cc:1397] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:42.751776 29316 raft_consensus.cc:348] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.751868 29316 raft_consensus.cc:374] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:42.751912 29316 raft_consensus.cc:729] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5dd98d1cf544629d330e75d9b2671e, State: Initialized, Role: FOLLOWER
I20260808 00:40:42.751955 29316 consensus_queue.cc:260] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.752039 29316 ts_tablet_manager.cc:1428] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 00:40:42.752135 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.748381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36375 (local address 127.0.0.1:57732)
0808 00:40:42.748421 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.748430 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.748460 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.749006 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.749007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.749009 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:42.749050 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.749052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.750066 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.750067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.750982 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.750985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.751883 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.751884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.751953 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.751974 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.752023 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.752068 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:40:42.752156 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.748468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57732 (local address 127.1.208.3:36375)
0808 00:40:42.748837 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.748841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.748851 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:40:42.748884 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.748885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.748899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.749000 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:42.749104 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.749994 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.751067 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.751805 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.751866 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.751929 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.752044 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.752079 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":88,"threads_started":1}
I20260808 00:40:42.752401 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.748391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38763 (local address 127.0.0.1:38112)
0808 00:40:42.748560 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.748567 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.748606 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.749014 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.749016 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.749018 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:42.749063 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.749064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.750338 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.750340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.751153 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.751156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.752196 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.752197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.752258 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.752261 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.752289 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.752338 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":124,"threads_started":1}
I20260808 00:40:42.752738 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.748645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38112 (local address 127.1.208.2:38763)
0808 00:40:42.748831 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.748837 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.748847 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:40:42.748888 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.748890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.748905 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.749131 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:42.749225 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.750224 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.751428 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.752099 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.752491 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.752559 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.752621 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.752659 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260808 00:40:42.752925 29862 tablet_service.cc:1467] Processing CreateTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 (DEFAULT_TABLE table=test-table [id=9aca1e94392c4cdf8cf73c7d361f0192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:42.753257 29862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e264bfd7e3349f186f0b9fa807e8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:42.753430 29629 tablet_service.cc:1467] Processing CreateTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 (DEFAULT_TABLE table=test-table [id=9aca1e94392c4cdf8cf73c7d361f0192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:40:42.753767 29629 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e264bfd7e3349f186f0b9fa807e8632. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:40:42.756695 30010 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Bootstrap starting.
I20260808 00:40:42.757035 30012 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Bootstrap starting.
I20260808 00:40:42.757440 30010 tablet_bootstrap.cc:654] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:42.757644 30012 tablet_bootstrap.cc:654] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Neither blocks nor log segments found. Creating new log.
I20260808 00:40:42.757936 30010 log.cc:824] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:42.758116 30012 log.cc:824] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Log is configured to *not* fsync() on all Append() calls
I20260808 00:40:42.758636 30010 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: No bootstrap required, opened a new log
I20260808 00:40:42.758711 30010 ts_tablet_manager.cc:1397] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:40:42.758819 30012 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: No bootstrap required, opened a new log
I20260808 00:40:42.758893 30012 ts_tablet_manager.cc:1397] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 00:40:42.760582 30010 raft_consensus.cc:348] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.760761 30010 raft_consensus.cc:374] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:42.760804 30010 raft_consensus.cc:729] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6a64ab9a55b45e1a7a47365aba77701, State: Initialized, Role: FOLLOWER
I20260808 00:40:42.760715 30012 raft_consensus.cc:348] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.760876 30012 raft_consensus.cc:374] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:40:42.760919 30012 raft_consensus.cc:729] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49a8d89f25494929a7f25d43e8f2e2ce, State: Initialized, Role: FOLLOWER
I20260808 00:40:42.760928 30010 consensus_queue.cc:260] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.761018 30012 consensus_queue.cc:260] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.761210 30010 ts_tablet_manager.cc:1428] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:40:42.761260 30012 ts_tablet_manager.cc:1428] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:40:42.761246 29987 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
I20260808 00:40:42.761314 29746 heartbeater.cc:503] Master 127.1.208.62:34455 was elected leader, sending a full tablet report...
W20260808 00:40:42.835600 29747 tablet.cc:2369] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:40:42.943939 29511 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:40:42.944105 29511 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.944445 29511 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:42.948173 30004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.944706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:56353 (local address 127.1.208.2:38763)
0808 00:40:42.944755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.944758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.944769 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:40:42.944802 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.944803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.944815 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.944878 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:42.945043 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.945815 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947204 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.947770 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947843 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.947854 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.948062 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.948099 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:40:42.948173 30001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.944714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:40997 (local address 127.1.208.3:36375)
0808 00:40:42.944756 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.944758 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.944834 (+ 76us) server_negotiation.cc:407] Connection header received
0808 00:40:42.944883 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.944884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.944895 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.944946 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:40:42.945072 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.945815 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947204 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.947769 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947843 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.947854 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.948062 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.948100 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:40:42.948199 29504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.944606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38763 (local address 127.1.208.1:56353)
0808 00:40:42.944651 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.944663 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.944722 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.944892 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.944893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.944895 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:42.944970 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.944973 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.945888 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.945889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.947121 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.947125 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.947858 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.947963 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.947991 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.948048 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.948099 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":5}
I20260808 00:40:42.948215 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.944613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36375 (local address 127.1.208.1:40997)
0808 00:40:42.944783 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:42.944791 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:40:42.944840 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:40:42.944953 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:40:42.944954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:40:42.944955 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:40:42.945013 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.945014 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.945888 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.945888 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.947124 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:40:42.947125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.947855 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.947856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:40:42.947963 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.947991 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 00:40:42.948048 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:40:42.948098 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":101,"thread_start_us":67,"threads_started":1}
I20260808 00:40:42.948560 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e264bfd7e3349f186f0b9fa807e8632" candidate_uuid: "ac5dd98d1cf544629d330e75d9b2671e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a64ab9a55b45e1a7a47365aba77701" is_pre_election: true
I20260808 00:40:42.948717 29900 raft_consensus.cc:2454] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac5dd98d1cf544629d330e75d9b2671e in term 0.
I20260808 00:40:42.948901 29662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e264bfd7e3349f186f0b9fa807e8632" candidate_uuid: "ac5dd98d1cf544629d330e75d9b2671e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" is_pre_election: true
I20260808 00:40:42.948951 29310 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e, c6a64ab9a55b45e1a7a47365aba77701; no voters:
I20260808 00:40:42.949051 29662 raft_consensus.cc:2454] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac5dd98d1cf544629d330e75d9b2671e in term 0.
I20260808 00:40:42.949106 29511 raft_consensus.cc:2781] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:40:42.949169 29511 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:40:42.949193 29511 raft_consensus.cc:3037] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:42.949937 29511 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.950059 29511 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 1 election: Requested vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
I20260808 00:40:42.950196 29662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e264bfd7e3349f186f0b9fa807e8632" candidate_uuid: "ac5dd98d1cf544629d330e75d9b2671e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
I20260808 00:40:42.950245 29900 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e264bfd7e3349f186f0b9fa807e8632" candidate_uuid: "ac5dd98d1cf544629d330e75d9b2671e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6a64ab9a55b45e1a7a47365aba77701"
I20260808 00:40:42.950268 29662 raft_consensus.cc:3037] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:42.950320 29900 raft_consensus.cc:3037] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:40:42.951004 29662 raft_consensus.cc:2454] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac5dd98d1cf544629d330e75d9b2671e in term 1.
I20260808 00:40:42.951009 29900 raft_consensus.cc:2454] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac5dd98d1cf544629d330e75d9b2671e in term 1.
I20260808 00:40:42.951161 29308 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49a8d89f25494929a7f25d43e8f2e2ce, ac5dd98d1cf544629d330e75d9b2671e; no voters:
I20260808 00:40:42.951309 29511 raft_consensus.cc:2781] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:40:42.951476 29511 raft_consensus.cc:686] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 LEADER]: Becoming Leader. State: Replica: ac5dd98d1cf544629d330e75d9b2671e, State: Running, Role: LEADER
I20260808 00:40:42.951556 29511 consensus_queue.cc:237] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:40:42.952348 29168 catalog_manager.cc:5495] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e reported cstate change: term changed from 0 to 1, leader changed from <none> to ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1). New cstate: current_term: 1 leader_uuid: "ac5dd98d1cf544629d330e75d9b2671e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } health_report { overall_health: HEALTHY } } }
W20260808 00:40:42.958032 29502 tablet.cc:2369] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:40:42.975457 29990 tablet.cc:2369] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:40:42.976279 29513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:42.974112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.1.208.1:40203)
0808 00:40:42.974162 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:40:42.974164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:40:42.974172 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:40:42.974205 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:40:42.974206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:40:42.974223 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:40:42.974281 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:40:42.974382 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.974864 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.975667 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:40:42.975889 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:40:42.975945 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:40:42.976051 (+ 106us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:40:42.976146 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:40:42.976193 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:40:42.976210 (+ 17us) server_negotiation.cc:299] Negotiation successful
0808 00:40:42.976227 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:40:42.977638 29900 raft_consensus.cc:1261] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Refusing update from remote peer ac5dd98d1cf544629d330e75d9b2671e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:42.977639 29662 raft_consensus.cc:1261] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Refusing update from remote peer ac5dd98d1cf544629d330e75d9b2671e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:40:42.977941 30021 consensus_queue.cc:1035] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:42.978071 29511 consensus_queue.cc:1035] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [LEADER]: Connected to new peer: Peer: permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:40:42.980638 30032 mvcc.cc:205] Tried to move back new op lower bound from 7316068937634193408 to 7316068937530228736. Current Snapshot: MvccSnapshot[applied={T|T < 7316068937634193408}]
I20260808 00:40:42.980752 30031 mvcc.cc:205] Tried to move back new op lower bound from 7316068937634193408 to 7316068937530228736. Current Snapshot: MvccSnapshot[applied={T|T < 7316068937634193408}]
I20260808 00:40:42.982981 29164 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:42.983068 29164 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56780:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:40:42.983727 29164 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Sending DeleteTablet for 3 replicas of tablet 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:40:42.984045 29382 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:33782
I20260808 00:40:42.984050 29862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:57732
I20260808 00:40:42.984061 29629 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e264bfd7e3349f186f0b9fa807e8632 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:40:42 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:38112
I20260808 00:40:42.984128 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 29237
I20260808 00:40:42.989966 30052 tablet_replica.cc:330] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: stopping tablet replica
I20260808 00:40:42.990012 30054 tablet_replica.cc:330] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: stopping tablet replica
I20260808 00:40:42.990099 30052 raft_consensus.cc:2229] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:42.990214 30052 raft_consensus.cc:2258] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:42.990304 30054 raft_consensus.cc:2229] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:40:42.990428 30054 raft_consensus.cc:2258] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:40:42.991595 30054 ts_tablet_manager.cc:1905] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:42.991595 30052 ts_tablet_manager.cc:1905] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:40:42.993321 30054 ts_tablet_manager.cc:1918] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:42.993359 30052 ts_tablet_manager.cc:1918] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:40:42.993418 30052 log.cc:1196] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:40:42.993520 30054 log.cc:1196] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:40:42.993738 30052 ts_tablet_manager.cc:1939] T 2e264bfd7e3349f186f0b9fa807e8632 P 49a8d89f25494929a7f25d43e8f2e2ce: Deleting consensus metadata
I20260808 00:40:42.993906 30054 ts_tablet_manager.cc:1939] T 2e264bfd7e3349f186f0b9fa807e8632 P c6a64ab9a55b45e1a7a47365aba77701: Deleting consensus metadata
I20260808 00:40:42.994184 29143 catalog_manager.cc:4841] TS 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): tablet 2e264bfd7e3349f186f0b9fa807e8632 (table test-table [id=9aca1e94392c4cdf8cf73c7d361f0192]) successfully deleted
I20260808 00:40:42.994366 29148 catalog_manager.cc:4841] TS c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): tablet 2e264bfd7e3349f186f0b9fa807e8632 (table test-table [id=9aca1e94392c4cdf8cf73c7d361f0192]) successfully deleted
W20260808 00:40:43.001247 29147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:40203 (error 108)
I20260808 00:40:43.001665 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 29508
I20260808 00:40:43.001781 29999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:40:43.001647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40203 (local address 127.0.0.1:33798)
0808 00:40:43.001700 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0808 00:40:43.001744 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:40:43.003373 29147 catalog_manager.cc:4584] TS ac5dd98d1cf544629d330e75d9b2671e (127.1.208.1:40203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e264bfd7e3349f186f0b9fa807e8632: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40203: connect: Connection refused (error 111)
I20260808 00:40:43.012158 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 29752
I20260808 00:40:43.030643 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 29112
I20260808 00:41:03.041036 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--webserver_interface=127.1.208.62
--webserver_port=43703
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:03.123155 30059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:03.123359 30059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:03.123382 30059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:03.124487 30059 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 00:41:03.124524 30059 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 00:41:03.124534 30059 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 00:41:03.124545 30059 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:03.124568 30059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:03.124591 30059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:03.124614 30059 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:03.124629 30059 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:03.126106 30059 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=43703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:03.126341 30059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:03.126554 30059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:41:03.131536 30059 server_base.cc:1034] running on GCE node
W20260808 00:41:03.131578 30073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.131575 30071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.131553 30068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:03.131981 30059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:03.132247 30059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:03.133351 30059 hybrid_clock.cc:648] HybridClock initialized: now 1786149663133338 us; error 32 us; skew 500 ppm
Aug 08 00:41:03 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:03.139390 30059 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:03.139858 30059 webserver.cc:460] Webserver started at http://127.1.208.62:43703/ using document root <none> and password file <none>
I20260808 00:41:03.140090 30059 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:03.140136 30059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:03.141578 30059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:03.142256 30086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.142428 30059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260808 00:41:03.142519 30059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "02969e29d1a4498e8caedd350797ea43"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "9e2eb81b69c2a80846a072e64f9a448c"
server_key_iv: "5d0984aab9b299fe2d5c29f79661ca96"
server_key_version: "encryptionkey@0"
I20260808 00:41:03.142871 30059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:03.161239 30059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:03.163184 30059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:03.163321 30059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:03.167752 30059 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34455
I20260808 00:41:03.167793 30183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34455 every 8 connection(s)
I20260808 00:41:03.168195 30059 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:03.169747 30184 sys_catalog.cc:263] Verifying existing consensus state
I20260808 00:41:03.170405 30184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap starting.
I20260808 00:41:03.177500 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 30059
I20260808 00:41:03.177802 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:40203
--local_ip_for_outbound_sockets=127.1.208.1
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=35041
--webserver_interface=127.1.208.1
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:03.179006 30184 log.cc:824] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:03.181820 30184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:41:03.182111 30184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Bootstrap complete.
I20260808 00:41:03.184296 30184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:41:03.184531 30184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Initialized, Role: FOLLOWER
I20260808 00:41:03.184690 30184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:41:03.184767 30184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:03.184818 30184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:03.184875 30184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 2 FOLLOWER]: Advancing to term 3
I20260808 00:41:03.185942 30184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:41:03.186039 30184 leader_election.cc:302] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02969e29d1a4498e8caedd350797ea43; no voters:
I20260808 00:41:03.186218 30184 leader_election.cc:288] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 00:41:03.186266 30191 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 00:41:03.186409 30191 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [term 3 LEADER]: Becoming Leader. State: Replica: 02969e29d1a4498e8caedd350797ea43, State: Running, Role: LEADER
I20260808 00:41:03.186496 30184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:03.186498 30191 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } }
I20260808 00:41:03.186853 30192 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:41:03.186967 30192 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:03.186848 30195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02969e29d1a4498e8caedd350797ea43. Latest consensus state: current_term: 3 leader_uuid: "02969e29d1a4498e8caedd350797ea43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02969e29d1a4498e8caedd350797ea43" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34455 } } }
I20260808 00:41:03.187150 30195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:03.187168 30202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:03.188077 30202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:03.188181 30202 catalog_manager.cc:1259] Loaded cluster ID: af4013a510ea4f65b977d14f08e2b2f7
I20260808 00:41:03.188213 30202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:03.189376 30202 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:03.189553 30202 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Loaded TSK: 0
I20260808 00:41:03.189597 30202 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 02969e29d1a4498e8caedd350797ea43: Loaded TSK: 1
F20260808 00:41:03.189826 30202 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f58d9c479ad google::LogMessage::Fail() at ??:0
@ 0x7f58d9c4a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f58d9c474de google::LogMessage::Flush() at ??:0
@ 0x7f58d9c48a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f58dc3b4104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f58dc3b1d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f58e1ba11cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f58e1ba148e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f58e1ba1594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f58e1b67c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f58e1b71ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f58dbb150d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f58dbb0a954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f58dc96e6db start_thread at ??:0
@ 0x7f58d874561f clone at ??:0
W20260808 00:41:03.280833 30190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:03.281030 30190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:03.281052 30190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:03.282203 30190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:03.282250 30190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:03.282276 30190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:03.283758 30190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:40203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=35041
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:03.283972 30190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:03.284193 30190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:03.289120 30276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.289166 30281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.289263 30275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:03.289597 30190 server_base.cc:1034] running on GCE node
I20260808 00:41:03.289785 30190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:03.290030 30190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:03.291173 30190 hybrid_clock.cc:648] HybridClock initialized: now 1786149663291160 us; error 31 us; skew 500 ppm
Aug 08 00:41:03 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:03.293629 30190 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:41:03.294070 30190 webserver.cc:460] Webserver started at http://127.1.208.1:35041/ using document root <none> and password file <none>
I20260808 00:41:03.294281 30190 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:03.294332 30190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:03.295673 30190 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:41:03.296404 30291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.296573 30190 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:03.296677 30190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ac5dd98d1cf544629d330e75d9b2671e"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "3daca426d662735ab4e1efeb18296ed0"
server_key_iv: "4fe8cdd56fcb57c099c9ec2d997a8dee"
server_key_version: "encryptionkey@0"
I20260808 00:41:03.297122 30190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:03.324373 30190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:03.326923 30190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:03.327098 30190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:03.327402 30190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:03.327934 30312 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 00:41:03.329551 30190 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 00:41:03.329598 30190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260808 00:41:03.329653 30190 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 00:41:03.330348 30190 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 00:41:03.330392 30190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260808 00:41:03.330472 30312 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap starting.
I20260808 00:41:03.338575 30190 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:40203
I20260808 00:41:03.338630 30514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:40203 every 8 connection(s)
I20260808 00:41:03.339116 30190 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:03.342132 30312 log.cc:824] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:03.344375 30312 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 00:41:03.344460 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 30190
I20260808 00:41:03.344659 30312 tablet_bootstrap.cc:492] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Bootstrap complete.
I20260808 00:41:03.344745 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:38763
--local_ip_for_outbound_sockets=127.1.208.2
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=33151
--webserver_interface=127.1.208.2
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:03.345644 30312 ts_tablet_manager.cc:1397] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.013s
I20260808 00:41:03.346799 30312 raft_consensus.cc:348] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:03.347112 30312 raft_consensus.cc:729] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac5dd98d1cf544629d330e75d9b2671e, State: Initialized, Role: FOLLOWER
I20260808 00:41:03.347288 30312 consensus_queue.cc:260] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:03.347572 30312 ts_tablet_manager.cc:1428] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
W20260808 00:41:03.398332 30518 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:03.339979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:58209)
0808 00:41:03.340210 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.340261 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:03.340881 (+ 620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:03.398105 (+ 57224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":115,"threads_started":1}
W20260808 00:41:03.398628 30515 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:34455 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:34455: Client connection negotiation failed: client connection to 127.1.208.62:34455: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 00:41:03.398996 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.398874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:33977)
0808 00:41:03.398923 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.398972 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:41:03.399374 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.399291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:56127)
0808 00:41:03.399330 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.399357 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 00:41:03.399488 30515 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 00:41:03.434280 30524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:03.434471 30524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:03.434494 30524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:03.435593 30524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:03.435632 30524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:03.435658 30524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:03.437088 30524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:38763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=33151
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:03.437314 30524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:03.437544 30524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:03.442284 30531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.442353 30530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.442276 30539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:03.442353 30524 server_base.cc:1034] running on GCE node
I20260808 00:41:03.442667 30524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:03.442868 30524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:03.444001 30524 hybrid_clock.cc:648] HybridClock initialized: now 1786149663443988 us; error 29 us; skew 500 ppm
Aug 08 00:41:03 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:03.446416 30524 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:41:03.446952 30524 webserver.cc:460] Webserver started at http://127.1.208.2:33151/ using document root <none> and password file <none>
I20260808 00:41:03.447196 30524 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:03.447244 30524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:03.448431 30524 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:03.448944 30548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.449090 30524 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.449165 30524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "49a8d89f25494929a7f25d43e8f2e2ce"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "06c22f4134f20e7d1682e7eea5c299f5"
server_key_iv: "c6602ee494ab67b980886536e567d818"
server_key_version: "encryptionkey@0"
I20260808 00:41:03.449546 30524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:03.457412 30524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:03.459067 30524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:03.459205 30524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:03.459424 30524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:03.459724 30524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:03.459762 30524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.459787 30524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:03.459815 30524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.466311 30524 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:38763
I20260808 00:41:03.466342 30751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:38763 every 8 connection(s)
I20260808 00:41:03.466717 30524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:41:03.467724 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.467347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:40521)
0808 00:41:03.467520 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.467588 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":80,"threads_started":1}
W20260808 00:41:03.468076 30752 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:34455 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:34455: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
I20260808 00:41:03.468360 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.468264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:48697)
0808 00:41:03.468313 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.468340 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260808 00:41:03.468590 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.468528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:51607)
0808 00:41:03.468557 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.468576 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260808 00:41:03.468714 30752 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:41:03.474793 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 30524
I20260808 00:41:03.475021 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:36375
--local_ip_for_outbound_sockets=127.1.208.3
--tserver_master_addrs=127.1.208.62:34455
--webserver_port=38179
--webserver_interface=127.1.208.3
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:03.556546 30755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:03.556730 30755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:03.556752 30755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:03.557844 30755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:03.557878 30755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:03.557901 30755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:03.559259 30755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:36375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=38179
--tserver_master_addrs=127.1.208.62:34455
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:03.559429 30755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:03.559602 30755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:41:03.564457 30755 server_base.cc:1034] running on GCE node
W20260808 00:41:03.564435 30768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.564450 30763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:03.564483 30764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:03.564832 30755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:03.565060 30755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:03.566193 30755 hybrid_clock.cc:648] HybridClock initialized: now 1786149663566159 us; error 56 us; skew 500 ppm
Aug 08 00:41:03 dist-test-slave-gjw7 krb5kdc[28148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:03.568607 30755 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:41:03.569054 30755 webserver.cc:460] Webserver started at http://127.1.208.3:38179/ using document root <none> and password file <none>
I20260808 00:41:03.569278 30755 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:03.569362 30755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:03.570801 30755 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:41:03.571380 30783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.571535 30755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:03.571611 30755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c6a64ab9a55b45e1a7a47365aba77701"
format_stamp: "Formatted at 2026-08-08 00:40:21 on dist-test-slave-gjw7"
server_key: "315ce6a5d45e196928025afb21314e12"
server_key_iv: "659be3d7c7a129a52f4c300c118f904f"
server_key_version: "encryptionkey@0"
I20260808 00:41:03.571961 30755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:03.591722 30755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:03.593698 30755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:03.593835 30755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:03.594064 30755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:03.594365 30755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:03.594399 30755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.594429 30755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:03.594446 30755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:03.600263 30755 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:36375
I20260808 00:41:03.600324 30979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:36375 every 8 connection(s)
I20260808 00:41:03.600723 30755 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:41:03.601853 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.601441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:37459)
0808 00:41:03.601609 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.601695 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
W20260808 00:41:03.602221 30980 heartbeater.cc:644] Failed to heartbeat to 127.1.208.62:34455 (0 consecutive failures): Network error: Failed to ping master at 127.1.208.62:34455: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
I20260808 00:41:03.602458 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.602386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:49505)
0808 00:41:03.602415 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.602440 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260808 00:41:03.602653 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:03.602605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:58585)
0808 00:41:03.602626 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:03.602642 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260808 00:41:03.602733 30980 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 00:41:03.610740 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 30755
I20260808 00:41:04.400055 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:04.399820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:53709)
0808 00:41:04.399948 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:04.399985 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":68,"threads_started":1}
I20260808 00:41:04.469254 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:04.469011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:60035)
0808 00:41:04.469145 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:04.469188 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260808 00:41:04.603178 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:04.602983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:49341)
0808 00:41:04.603098 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:04.603130 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260808 00:41:05.067137 30989 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:05.067281 30989 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:05.067647 30989 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
Aug 08 00:41:05 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.3@KRBTEST.COM
Aug 08 00:41:05 dist-test-slave-gjw7 krb5kdc[28148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149663, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.2@KRBTEST.COM
I20260808 00:41:05.080307 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.067904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:47167 (local address 127.1.208.3:36375)
0808 00:41:05.068096 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:05.068108 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:05.068142 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:41:05.068716 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:05.068720 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:05.068783 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:05.068943 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:05.069975 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.071392 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.072944 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.073618 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.073730 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:05.076623 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:05.076628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:05.076628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:05.076642 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:05.078618 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079133 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:05.079136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:05.079137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:05.079273 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079459 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:05.079460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:05.079460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:05.079697 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:05.079777 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:05.080049 (+ 272us) server_negotiation.cc:299] Negotiation successful
0808 00:41:05.080178 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":77,"threads_started":1}
I20260808 00:41:05.080307 30998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.067952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:60533 (local address 127.1.208.2:38763)
0808 00:41:05.068095 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:05.068109 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:05.068134 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:41:05.068717 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:05.068720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:05.068783 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:05.068943 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:05.069975 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.071392 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.072941 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.073618 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.073730 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:05.076952 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:05.076956 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:05.076956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:05.076968 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:05.078745 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079111 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:05.079116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:05.079117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:05.079219 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079435 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:05.079436 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:05.079436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:05.079694 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:05.079767 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:05.080046 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 00:41:05.080180 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":42,"threads_started":1}
I20260808 00:41:05.080307 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.067755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:36375 (local address 127.1.208.1:47167)
0808 00:41:05.067918 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:05.067929 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:05.067992 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:05.069028 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:05.069032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:05.069542 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:05.069829 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:05.069838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.071533 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.071534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:05.072842 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:05.072846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.073725 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.073727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:05.073891 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:05.074511 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:05.074523 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:05.076501 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:05.078748 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:05.078755 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:05.078758 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:05.079038 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:05.079343 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:05.079344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:05.079405 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:05.079766 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:05.079766 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:05.079891 (+ 125us) client_negotiation.cc:769] Sending connection context
0808 00:41:05.079979 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:41:05.080185 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":62,"thread_start_us":49,"threads_started":1}
I20260808 00:41:05.080381 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.067810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:38763 (local address 127.1.208.1:60533)
0808 00:41:05.067983 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:05.067992 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:05.068040 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:05.069028 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:05.069032 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:05.069542 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:05.069829 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:05.069838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.071521 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.071522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:05.072842 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:05.072846 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:05.073726 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:05.073727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:05.073891 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:05.074551 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:05.074555 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:05.076851 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:05.078813 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:05.078815 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:05.078815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:05.079040 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:05.079277 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:05.079278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:05.079278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:05.079378 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:05.079758 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:05.079766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:05.079890 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 00:41:05.079978 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 00:41:05.080260 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"mutex_wait_us":218,"thread_start_us":111,"threads_started":1}
W20260808 00:41:05.081244 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:05.081697 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:05.081748 30304 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:05.081864 30989 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:05.400655 30992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.400484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:33497)
0808 00:41:05.400554 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:05.400603 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 00:41:05.469907 31003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.469686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:42039)
0808 00:41:05.469810 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:05.469856 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":77,"threads_started":1}
I20260808 00:41:05.603754 31004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:05.603538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:50865)
0808 00:41:05.603662 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:05.603709 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":54,"threads_started":1}
I20260808 00:41:06.401265 31006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:06.401060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:50689)
0808 00:41:06.401180 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:06.401214 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":68,"threads_started":1}
I20260808 00:41:06.470533 31008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:06.470345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:53931)
0808 00:41:06.470449 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:06.470486 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":55,"threads_started":1}
I20260808 00:41:06.604393 31009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:06.604180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:59061)
0808 00:41:06.604320 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:06.604355 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
I20260808 00:41:06.851184 31010 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:06.851284 31010 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:06.851454 31010 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
W20260808 00:41:06.851873 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:06.852010 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:06.852078 30304 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:06.852142 31010 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:07.401898 31015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:07.401681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:48909)
0808 00:41:07.401808 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:07.401846 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260808 00:41:07.471118 31017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:07.470919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:45749)
0808 00:41:07.471036 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:07.471073 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260808 00:41:07.605044 31018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:07.604817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:33499)
0808 00:41:07.604964 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:07.605002 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260808 00:41:08.402629 31022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:08.402415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:51107)
0808 00:41:08.402545 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:08.402581 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":68,"threads_started":1}
I20260808 00:41:08.471733 31024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:08.471519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:52219)
0808 00:41:08.471653 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:08.471686 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260808 00:41:08.605716 31028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:08.605477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:46243)
0808 00:41:08.605617 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:08.605656 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260808 00:41:09.130723 31029 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:09.130831 31029 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:09.131008 31029 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
W20260808 00:41:09.131398 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:09.131485 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:09.131532 30294 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:09.131613 31029 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:09.403287 31032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:09.403038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:42841)
0808 00:41:09.403200 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:09.403238 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260808 00:41:09.472406 31033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:09.472159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:44255)
0808 00:41:09.472309 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:09.472360 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260808 00:41:09.606418 31034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:09.606165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:57837)
0808 00:41:09.606335 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:09.606374 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260808 00:41:10.403951 31036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:10.403701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:43415)
0808 00:41:10.403865 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:10.403901 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":74,"threads_started":1}
I20260808 00:41:10.473052 31037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:10.472803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:56551)
0808 00:41:10.472969 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:10.473007 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":79,"threads_started":1}
I20260808 00:41:10.607136 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:10.606897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:51757)
0808 00:41:10.607048 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:10.607088 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":71,"threads_started":1}
I20260808 00:41:11.404625 31041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:11.404368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:52143)
0808 00:41:11.404520 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:11.404564 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":66,"threads_started":1}
I20260808 00:41:11.474164 31043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:11.473399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:37541)
0808 00:41:11.474065 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:11.474109 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":600,"threads_started":1}
I20260808 00:41:11.607828 31044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:11.607604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:43369)
0808 00:41:11.607743 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:11.607781 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260808 00:41:12.405247 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:12.405033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:37267)
0808 00:41:12.405158 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:12.405194 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260808 00:41:12.474846 31050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:12.474620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:33015)
0808 00:41:12.474762 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:12.474799 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260808 00:41:12.608497 31052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:12.608266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:51741)
0808 00:41:12.608413 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:12.608452 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":86,"threads_started":1}
I20260808 00:41:13.405937 31053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:13.405702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:52313)
0808 00:41:13.405845 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:13.405885 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 00:41:13.422031 31058 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:13.422120 31058 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:13.422290 31058 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
W20260808 00:41:13.422667 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:13.422757 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:13.422827 30304 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:13.422909 31058 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:13.475443 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:13.475202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:48341)
0808 00:41:13.475364 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:13.475399 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":82,"threads_started":1}
I20260808 00:41:13.609186 31060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:13.608938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:42069)
0808 00:41:13.609087 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:13.609125 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260808 00:41:14.406550 31063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:14.406320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:57641)
0808 00:41:14.406462 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:14.406500 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":68,"threads_started":1}
I20260808 00:41:14.476050 31064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:14.475824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:49897)
0808 00:41:14.475967 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:14.476002 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260808 00:41:14.609833 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:14.609633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:50741)
0808 00:41:14.609754 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:14.609789 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260808 00:41:15.317986 31069 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:15.318099 31069 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:15.318284 31069 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
W20260808 00:41:15.318691 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:15.318799 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:15.318863 30294 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:15.318929 31069 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:15.407141 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:15.406921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:45853)
0808 00:41:15.407055 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:15.407090 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
I20260808 00:41:15.476708 31071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:15.476499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:47731)
0808 00:41:15.476629 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:15.476663 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260808 00:41:15.610491 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:15.610283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:54133)
0808 00:41:15.610415 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:15.610446 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260808 00:41:16.407781 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:16.407575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:51667)
0808 00:41:16.407694 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:16.407731 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
I20260808 00:41:16.477448 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:16.477170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:50369)
0808 00:41:16.477344 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:16.477387 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":83,"threads_started":1}
I20260808 00:41:16.611143 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:16.610928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:58073)
0808 00:41:16.611064 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:16.611098 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":72,"threads_started":1}
I20260808 00:41:17.408429 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:17.408196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:36497)
0808 00:41:17.408338 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:17.408378 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260808 00:41:17.478050 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:17.477842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:50805)
0808 00:41:17.477967 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:17.478003 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260808 00:41:17.611802 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:17.611556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:52883)
0808 00:41:17.611717 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:17.611754 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260808 00:41:17.854660 31084 raft_consensus.cc:482] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:17.854777 31084 raft_consensus.cc:504] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c6a64ab9a55b45e1a7a47365aba77701" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 36375 } } peers { permanent_uuid: "49a8d89f25494929a7f25d43e8f2e2ce" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 38763 } } peers { permanent_uuid: "ac5dd98d1cf544629d330e75d9b2671e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40203 } }
I20260808 00:41:17.854954 31084 leader_election.cc:288] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375), 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763)
W20260808 00:41:17.855297 30304 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c6a64ab9a55b45e1a7a47365aba77701 (127.1.208.3:36375): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
W20260808 00:41:17.855378 30294 leader_election.cc:341] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 49a8d89f25494929a7f25d43e8f2e2ce (127.1.208.2:38763): Not found: Tablet not found: 2e264bfd7e3349f186f0b9fa807e8632
I20260808 00:41:17.855437 30294 leader_election.cc:302] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ac5dd98d1cf544629d330e75d9b2671e; no voters: 49a8d89f25494929a7f25d43e8f2e2ce, c6a64ab9a55b45e1a7a47365aba77701
I20260808 00:41:17.855520 31084 raft_consensus.cc:2726] T 2e264bfd7e3349f186f0b9fa807e8632 P ac5dd98d1cf544629d330e75d9b2671e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 00:41:18.409085 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:18.408888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.1:52889)
0808 00:41:18.409007 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:18.409036 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260808 00:41:18.478700 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:18.478504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.2:51045)
0808 00:41:18.478622 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:18.478652 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260808 00:41:18.612452 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:18.612256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34455 (local address 127.1.208.3:35913)
0808 00:41:18.612380 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:18.612412 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.62:34455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260808 00:41:18.613233 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 30190
I20260808 00:41:18.627944 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 30524
I20260808 00:41:18.643846 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 30755
2026-08-08T00:41:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58617 ms)
[----------] 30 tests from SecurityITest (238531 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:18 dist-test-slave-gjw7 krb5kdc[31094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:18 dist-test-slave-gjw7 krb5kdc[31094](info): set up 2 sockets
Aug 08 00:41:18 dist-test-slave-gjw7 krb5kdc[31094](info): commencing operation
krb5kdc: starting...
W20260808 00:41:19.809083 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:19 dist-test-slave-gjw7 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:19.925901 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:39607
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:39607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:20.007244 31114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:20.007463 31114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:20.007493 31114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:20.008704 31114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:20.008750 31114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:20.008770 31114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:20.008787 31114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:20.008801 31114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:20.010310 31114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:39607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:39607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:20.010563 31114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:20.010775 31114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:20.015588 31128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.015627 31127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.015627 31130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.015772 31114 server_base.cc:1034] running on GCE node
I20260808 00:41:20.016074 31114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:20.016355 31114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:20.017494 31114 hybrid_clock.cc:648] HybridClock initialized: now 1786149680017474 us; error 32 us; skew 500 ppm
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:20.023137 31114 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:20.023557 31114 webserver.cc:460] Webserver started at http://127.1.208.62:44907/ using document root <none> and password file <none>
I20260808 00:41:20.023757 31114 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:20.023797 31114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:20.023892 31114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:20.024920 31114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "dff442e6601b452b8de138122a1dbcba"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "4ef28397c9d80f321deb230a97b78920"
server_key_iv: "b33a0b97d179c1e458eb1059e577abeb"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.025242 31114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "dff442e6601b452b8de138122a1dbcba"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "4ef28397c9d80f321deb230a97b78920"
server_key_iv: "b33a0b97d179c1e458eb1059e577abeb"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.026522 31114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:20.027241 31140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.027416 31114 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:20.027504 31114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "dff442e6601b452b8de138122a1dbcba"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "4ef28397c9d80f321deb230a97b78920"
server_key_iv: "b33a0b97d179c1e458eb1059e577abeb"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.027560 31114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:20.038856 31114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:20.040969 31114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:20.041080 31114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:20.044936 31114 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:39607
I20260808 00:41:20.044978 31236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:39607 every 8 connection(s)
I20260808 00:41:20.045428 31114 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:20.046315 31242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:20.048622 31242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: Bootstrap starting.
I20260808 00:41:20.049196 31242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:20.049494 31242 log.cc:824] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:20.050297 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 31114
I20260808 00:41:20.050414 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:20.050441 31242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: No bootstrap required, opened a new log
I20260808 00:41:20.050633 1856 external_mini_cluster.cc:1428] Setting key 64d8a9bde3f2251837c10920bd9da30a
I20260808 00:41:20.052804 31242 raft_consensus.cc:348] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } }
I20260808 00:41:20.052959 31242 raft_consensus.cc:374] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:20.052984 31242 raft_consensus.cc:729] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dff442e6601b452b8de138122a1dbcba, State: Initialized, Role: FOLLOWER
I20260808 00:41:20.053102 31242 consensus_queue.cc:260] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } }
I20260808 00:41:20.053174 31242 raft_consensus.cc:388] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:20.053202 31242 raft_consensus.cc:482] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:20.053253 31242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:20.053853 31242 raft_consensus.cc:504] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } }
I20260808 00:41:20.053951 31242 leader_election.cc:302] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dff442e6601b452b8de138122a1dbcba; no voters:
I20260808 00:41:20.054129 31242 leader_election.cc:288] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:20.054162 31250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:20.054332 31250 raft_consensus.cc:686] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [term 1 LEADER]: Becoming Leader. State: Replica: dff442e6601b452b8de138122a1dbcba, State: Running, Role: LEADER
I20260808 00:41:20.054473 31242 sys_catalog.cc:564] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:20.054486 31250 consensus_queue.cc:237] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } }
I20260808 00:41:20.054881 31250 sys_catalog.cc:455] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [sys.catalog]: SysCatalogTable state changed. Reason: New leader dff442e6601b452b8de138122a1dbcba. Latest consensus state: current_term: 1 leader_uuid: "dff442e6601b452b8de138122a1dbcba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } } }
I20260808 00:41:20.055243 31253 sys_catalog.cc:455] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dff442e6601b452b8de138122a1dbcba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dff442e6601b452b8de138122a1dbcba" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39607 } } }
I20260808 00:41:20.055334 31250 sys_catalog.cc:458] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:20.055393 31253 sys_catalog.cc:458] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:20.055943 31267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:20.056624 31267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:20.058113 31267 catalog_manager.cc:1347] Generated new cluster ID: 9616d22bd023474baca7678497a37902
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:20.058172 31267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:20.062538 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.051154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.1.208.62:39607)
0808 00:41:20.051318 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.051349 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.051374 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:41:20.051818 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.051825 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.051875 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.051981 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:20.052802 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.054099 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.055109 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.055695 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.055782 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.058734 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:20.058739 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:20.058740 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:20.058751 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:20.060725 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.061120 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.061123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.061124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.061226 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.061483 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.061484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.061485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.061787 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:20.061914 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.062185 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.062338 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260808 00:41:20.075029 31267 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:20.075567 31267 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:20.080938 31267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: Generated new TSK 0
I20260808 00:41:20.081108 31267 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:41:20.109886 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:20.209273 31288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:20.209519 31288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:20.209553 31288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:20.210714 31288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:20.210755 31288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:20.210790 31288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:20.212257 31288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:20.212486 31288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:20.212694 31288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:20.217506 31296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.217516 31299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.217537 31295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.217716 31288 server_base.cc:1034] running on GCE node
I20260808 00:41:20.217912 31288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:20.218132 31288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:20.219262 31288 hybrid_clock.cc:648] HybridClock initialized: now 1786149680219252 us; error 29 us; skew 500 ppm
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:20.221814 31288 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:41:20.222244 31288 webserver.cc:460] Webserver started at http://127.1.208.1:40895/ using document root <none> and password file <none>
I20260808 00:41:20.222440 31288 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:20.222477 31288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:20.222568 31288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:20.223558 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "b3dac82dbac7467791932af21323572e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "2df3fbe3521687fa4ee2e9d21d37c8ec"
server_key_iv: "12c684381af33bbf62a666fca0327f7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.223865 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "b3dac82dbac7467791932af21323572e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "2df3fbe3521687fa4ee2e9d21d37c8ec"
server_key_iv: "12c684381af33bbf62a666fca0327f7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.225016 31288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:41:20.225735 31311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.225878 31288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.225955 31288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "b3dac82dbac7467791932af21323572e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "2df3fbe3521687fa4ee2e9d21d37c8ec"
server_key_iv: "12c684381af33bbf62a666fca0327f7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.226011 31288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:20.244289 31288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:20.245997 31288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:20.246104 31288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:20.246289 31288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:20.246544 31288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:20.246572 31288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.246600 31288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:20.246616 31288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.251969 31288 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:43573
I20260808 00:41:20.252031 31526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:43573 every 8 connection(s)
I20260808 00:41:20.252374 31288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:20.255595 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 31288
I20260808 00:41:20.255718 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:41:20.255936 1856 external_mini_cluster.cc:1428] Setting key 07d9d1c9783cadd064c8c3f8371de2c6
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:20.262620 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.253246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:48925 (local address 127.1.208.62:39607)
0808 00:41:20.253345 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.253347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.253588 (+ 241us) server_negotiation.cc:407] Connection header received
0808 00:41:20.254238 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.254240 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.254253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.254311 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:20.255567 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.256176 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.257461 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.257657 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.257709 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.259825 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:20.259826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:20.259827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:20.259833 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:20.261169 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.261641 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.261641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.261642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.261693 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.261950 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.261950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.261951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.262049 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:20.262095 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.262468 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.262549 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 00:41:20.262785 31530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.253223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:39607 (local address 127.1.208.1:48925)
0808 00:41:20.253428 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.253472 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.254118 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.254397 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.254452 (+ 55us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.255214 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:20.255424 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.255432 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.256253 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.256254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.257390 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.257394 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.257729 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.257730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.257867 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.258392 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:20.258402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:20.259739 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:20.261252 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.261260 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.261262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.261570 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.261748 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.261748 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.261749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.261888 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.262103 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:20.262109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:20.262271 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.262394 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.262562 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":110,"threads_started":1}
I20260808 00:41:20.263276 31527 heartbeater.cc:347] Connected to a master server at 127.1.208.62:39607
I20260808 00:41:20.263370 31527 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:20.263540 31527 heartbeater.cc:511] Master 127.1.208.62:39607 requested a full tablet report, sending...
I20260808 00:41:20.264053 31169 ts_manager.cc:194] Registered new tserver with Master: b3dac82dbac7467791932af21323572e (127.1.208.1:43573)
I20260808 00:41:20.264658 31169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:48925
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:41:20.284138 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:20.367020 31537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:20.367240 31537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:20.367274 31537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:20.368389 31537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:20.368428 31537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:20.368462 31537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:20.369907 31537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:20.370147 31537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:20.370365 31537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:20.375118 31549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.375131 31544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.375217 31546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.375324 31537 server_base.cc:1034] running on GCE node
I20260808 00:41:20.375532 31537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:20.375761 31537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:20.376909 31537 hybrid_clock.cc:648] HybridClock initialized: now 1786149680376890 us; error 30 us; skew 500 ppm
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:20.379475 31537 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:41:20.379895 31537 webserver.cc:460] Webserver started at http://127.1.208.2:38227/ using document root <none> and password file <none>
I20260808 00:41:20.380084 31537 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:20.380123 31537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:20.380223 31537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:20.381213 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "4a02eded021e47f694e898a078d1ce6e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1ccd8be73b2e3b3966907c6991560d35"
server_key_iv: "b10a47e065fd987086e18399b2a38fa7"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.381572 31537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "4a02eded021e47f694e898a078d1ce6e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1ccd8be73b2e3b3966907c6991560d35"
server_key_iv: "b10a47e065fd987086e18399b2a38fa7"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.382750 31537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:20.383427 31562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.383572 31537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.383659 31537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "4a02eded021e47f694e898a078d1ce6e"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1ccd8be73b2e3b3966907c6991560d35"
server_key_iv: "b10a47e065fd987086e18399b2a38fa7"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.383723 31537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:20.391769 31537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:20.393570 31537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:20.393673 31537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:20.393844 31537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:20.394088 31537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:20.394117 31537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.394136 31537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:20.394155 31537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.399824 31537 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:35439
I20260808 00:41:20.399859 31773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:35439 every 8 connection(s)
I20260808 00:41:20.400137 31537 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:20.408421 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.400791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:55309 (local address 127.1.208.62:39607)
0808 00:41:20.400852 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.400854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.401016 (+ 162us) server_negotiation.cc:407] Connection header received
0808 00:41:20.401546 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.401548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.401559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.401613 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:20.402467 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.402958 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.403937 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.404132 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.404184 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.406141 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:20.406142 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:20.406143 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:20.406149 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:20.407333 (+ 1184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.407667 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.407668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.407668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.407715 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.407897 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.407898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.407898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.407991 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:20.408036 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.408279 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.408353 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:41:20.408538 31778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.400756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:39607 (local address 127.1.208.2:55309)
0808 00:41:20.400906 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.400942 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.401453 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.401664 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.401668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.402215 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:20.402394 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.402400 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.403019 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.403020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.403861 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.403864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.404190 (+ 326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.404190 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.404288 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.404727 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:20.404736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:20.406062 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:20.407410 (+ 1348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.407416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.407417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.407615 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.407762 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.407762 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.407763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.407855 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.408032 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:20.408034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:20.408136 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.408243 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.408363 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260808 00:41:20.409025 31775 heartbeater.cc:347] Connected to a master server at 127.1.208.62:39607
I20260808 00:41:20.409112 31775 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:20.409307 31775 heartbeater.cc:511] Master 127.1.208.62:39607 requested a full tablet report, sending...
I20260808 00:41:20.409257 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 31537
I20260808 00:41:20.409394 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:20.409631 1856 external_mini_cluster.cc:1428] Setting key 36e7a1cd110411134cba5643bb7c271f
I20260808 00:41:20.409732 31169 ts_manager.cc:194] Registered new tserver with Master: 4a02eded021e47f694e898a078d1ce6e (127.1.208.2:35439)
I20260808 00:41:20.410128 31169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:55309
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:41:20.435501 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:44277
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:20.516394 31782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:20.516590 31782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:20.516620 31782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:20.517751 31782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:20.517791 31782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:20.517832 31782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:20.519261 31782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44277
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:39607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:20.519496 31782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:20.519701 31782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:20.524784 31797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.524869 31782 server_base.cc:1034] running on GCE node
W20260808 00:41:20.524748 31795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.524789 31794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.525218 31782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:20.525463 31782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:20.526599 31782 hybrid_clock.cc:648] HybridClock initialized: now 1786149680526586 us; error 35 us; skew 500 ppm
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:20.529026 31782 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:41:20.529520 31782 webserver.cc:460] Webserver started at http://127.1.208.3:44975/ using document root <none> and password file <none>
I20260808 00:41:20.529822 31782 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:20.529882 31782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:20.530014 31782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:20.531117 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "2a30fe6b9f3d4c8a87513d454155a3da"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "22ad49838bc69f14273af4728a93be41"
server_key_iv: "4663d49bbd3f435b71b76cf26a0d4aba"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.531478 31782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "2a30fe6b9f3d4c8a87513d454155a3da"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "22ad49838bc69f14273af4728a93be41"
server_key_iv: "4663d49bbd3f435b71b76cf26a0d4aba"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.532765 31782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:20.533497 31806 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.533638 31782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:20.533722 31782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "2a30fe6b9f3d4c8a87513d454155a3da"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "22ad49838bc69f14273af4728a93be41"
server_key_iv: "4663d49bbd3f435b71b76cf26a0d4aba"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.533784 31782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:20.542224 31782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:20.543848 31782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:20.543954 31782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:20.544130 31782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:20.544395 31782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:20.544427 31782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.544447 31782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:20.544464 31782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.550261 31782 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:39649
I20260808 00:41:20.550320 32016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:39649 every 8 connection(s)
I20260808 00:41:20.550602 31782 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:41:20 dist-test-slave-gjw7 krb5kdc[31094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149680, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:20.559309 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.551210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:60983 (local address 127.1.208.62:39607)
0808 00:41:20.551252 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.551254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.551455 (+ 201us) server_negotiation.cc:407] Connection header received
0808 00:41:20.551954 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.551956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.551969 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.552019 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:20.552936 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.553486 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.554531 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.554804 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.554856 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.556917 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:20.556919 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:20.556919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:20.556926 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:20.558049 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.558411 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.558413 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.558413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.558469 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.558771 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.558773 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.558773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.558897 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:20.558960 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.559177 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.559242 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:20.559454 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.551189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:39607 (local address 127.1.208.3:60983)
0808 00:41:20.551335 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.551369 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.551854 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.552091 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.552095 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.552652 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:20.552836 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.552842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.553565 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.553566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.554451 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.554453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.554886 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.554888 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.555009 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.555429 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:20.555438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:20.556835 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:20.558122 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.558129 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.558131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.558345 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.558522 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:20.558523 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:20.558523 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:20.558649 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:20.558959 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:20.558965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:20.559078 (+ 113us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.559157 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.559281 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260808 00:41:20.559908 32017 heartbeater.cc:347] Connected to a master server at 127.1.208.62:39607
I20260808 00:41:20.560002 32017 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:20.560160 32017 heartbeater.cc:511] Master 127.1.208.62:39607 requested a full tablet report, sending...
I20260808 00:41:20.560508 31168 ts_manager.cc:194] Registered new tserver with Master: 2a30fe6b9f3d4c8a87513d454155a3da (127.1.208.3:39649)
I20260808 00:41:20.560654 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 31782
I20260808 00:41:20.560746 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:20.560931 1856 external_mini_cluster.cc:1428] Setting key 088763a9a1ecb53e0d10de58a0b9946b
I20260808 00:41:20.560946 31168 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:60983
I20260808 00:41:20.561738 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:20.568059 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.563057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.1.208.62:39607)
0808 00:41:20.563100 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.563102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.563177 (+ 75us) server_negotiation.cc:407] Connection header received
0808 00:41:20.563274 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.563275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.563283 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.563320 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:20.563788 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.564259 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.564924 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.565097 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.565142 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.566143 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:20.566144 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:20.566144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:20.566150 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:20.567193 (+ 1043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.567506 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.567506 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.567507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.567552 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:20.567659 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:20.567660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:20.567660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:20.567790 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:20.567862 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.567932 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.567991 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:41:20.569425 31168 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:20.571395 31168 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:20.577596 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43573 (local address 127.0.0.1:57136)
0808 00:41:20.574090 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.574097 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.574130 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.574347 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.574352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.574354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:20.574424 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.574426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.575402 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.575403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.576513 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.576516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.577259 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.577260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.577379 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.577393 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.577467 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.577517 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":53,"threads_started":1}
I20260808 00:41:20.577613 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57136 (local address 127.1.208.1:43573)
0808 00:41:20.574140 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.574144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.574153 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:20.574189 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.574191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.574202 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.574334 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:20.574531 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.575315 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.576585 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.577175 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.577248 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.577340 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.577487 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.577533 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":94,"threads_started":1}
I20260808 00:41:20.578081 32041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35439 (local address 127.0.0.1:59666)
0808 00:41:20.573913 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.573954 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.574009 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.574498 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.574498 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.574500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:20.574538 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.574539 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.575838 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.575839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.576799 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.576801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.577870 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.577871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.577922 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.577925 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.577974 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.578016 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":64,"threads_started":1}
I20260808 00:41:20.578123 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.1.208.2:35439)
0808 00:41:20.574188 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.574213 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.574222 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:20.574259 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.574261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.574277 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.574348 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:20.574599 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.575729 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.576882 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.577589 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.577779 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.577838 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.578006 (+ 168us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.578046 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":97,"threads_started":1}
I20260808 00:41:20.578531 31397 tablet_service.cc:1467] Processing CreateTablet for tablet b22784433ca84140836393da15de552a (DEFAULT_TABLE table=test-table [id=28b8062ee91b4a069ba72b82dab4573d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:20.578867 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:39649 (local address 127.0.0.1:59520)
0808 00:41:20.573920 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.573955 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.574013 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.574646 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.574647 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.574649 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:20.574673 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.574674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.576868 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.576868 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.577708 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.577712 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.578647 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.578648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.578708 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.578711 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.578761 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.578800 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":3,"thread_start_us":59,"threads_started":1}
I20260808 00:41:20.578958 31397 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22784433ca84140836393da15de552a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:20.578896 31639 tablet_service.cc:1467] Processing CreateTablet for tablet b22784433ca84140836393da15de552a (DEFAULT_TABLE table=test-table [id=28b8062ee91b4a069ba72b82dab4573d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:20.579097 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.573692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59520 (local address 127.1.208.3:39649)
0808 00:41:20.574506 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.574510 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.574518 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:20.574552 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.574554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.574568 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.574654 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:20.574725 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.575748 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.577796 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.578488 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.578555 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.578611 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.578976 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.579021 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":105,"threads_started":1}
I20260808 00:41:20.579236 31639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22784433ca84140836393da15de552a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:20.581694 32044 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Bootstrap starting.
I20260808 00:41:20.582320 32045 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Bootstrap starting.
I20260808 00:41:20.582321 31893 tablet_service.cc:1467] Processing CreateTablet for tablet b22784433ca84140836393da15de552a (DEFAULT_TABLE table=test-table [id=28b8062ee91b4a069ba72b82dab4573d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:20.582414 32044 tablet_bootstrap.cc:654] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:20.582610 31893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b22784433ca84140836393da15de552a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:20.582994 32045 tablet_bootstrap.cc:654] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:20.583003 32044 log.cc:824] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:20.583488 32045 log.cc:824] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:20.583799 32044 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: No bootstrap required, opened a new log
I20260808 00:41:20.583863 32044 ts_tablet_manager.cc:1397] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:41:20.584244 32045 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: No bootstrap required, opened a new log
I20260808 00:41:20.584314 32045 ts_tablet_manager.cc:1397] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:20.585464 32049 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Bootstrap starting.
I20260808 00:41:20.585817 32044 raft_consensus.cc:348] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.586002 32044 raft_consensus.cc:374] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:20.586047 32044 raft_consensus.cc:729] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a02eded021e47f694e898a078d1ce6e, State: Initialized, Role: FOLLOWER
I20260808 00:41:20.586087 32049 tablet_bootstrap.cc:654] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:20.586194 32044 consensus_queue.cc:260] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.586337 32045 raft_consensus.cc:348] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.586519 32045 raft_consensus.cc:374] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:20.586529 32044 ts_tablet_manager.cc:1428] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:41:20.586562 32045 raft_consensus.cc:729] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dac82dbac7467791932af21323572e, State: Initialized, Role: FOLLOWER
I20260808 00:41:20.586648 31775 heartbeater.cc:503] Master 127.1.208.62:39607 was elected leader, sending a full tablet report...
I20260808 00:41:20.586653 32049 log.cc:824] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:20.586694 32045 consensus_queue.cc:260] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.586966 32045 ts_tablet_manager.cc:1428] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:41:20.587042 31527 heartbeater.cc:503] Master 127.1.208.62:39607 was elected leader, sending a full tablet report...
I20260808 00:41:20.587545 32049 tablet_bootstrap.cc:492] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: No bootstrap required, opened a new log
I20260808 00:41:20.587612 32049 ts_tablet_manager.cc:1397] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:20.588987 32049 raft_consensus.cc:348] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.589114 32049 raft_consensus.cc:374] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:20.589146 32049 raft_consensus.cc:729] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a30fe6b9f3d4c8a87513d454155a3da, State: Initialized, Role: FOLLOWER
I20260808 00:41:20.589250 32049 consensus_queue.cc:260] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.589511 32049 ts_tablet_manager.cc:1428] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:41:20.589629 32017 heartbeater.cc:503] Master 127.1.208.62:39607 was elected leader, sending a full tablet report...
W20260808 00:41:20.650660 31777 tablet.cc:2369] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:20.751960 32056 raft_consensus.cc:482] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:20.752118 32056 raft_consensus.cc:504] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.752501 32056 leader_election.cc:288] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a02eded021e47f694e898a078d1ce6e (127.1.208.2:35439), b3dac82dbac7467791932af21323572e (127.1.208.1:43573)
W20260808 00:41:20.753331 31529 tablet.cc:2369] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:20.755555 32021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.752709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43573 (local address 127.1.208.3:50307)
0808 00:41:20.752758 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.752769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.752818 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.753032 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.753033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.753035 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:20.753111 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.753114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.753932 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.753933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.754719 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.754721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.755305 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.755306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.755391 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.755394 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.755440 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.755483 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:41:20.755555 32042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.752793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:50307 (local address 127.1.208.1:43573)
0808 00:41:20.752847 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.752850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.752896 (+ 46us) server_negotiation.cc:407] Connection header received
0808 00:41:20.752933 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.752934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.752949 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.753010 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:20.753190 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.753853 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.754808 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.755236 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.755296 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.755308 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.755452 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.755490 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 00:41:20.755877 32040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.752822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:59647 (local address 127.1.208.2:35439)
0808 00:41:20.752876 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.752878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.753102 (+ 224us) server_negotiation.cc:407] Connection header received
0808 00:41:20.753139 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.753140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.753154 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.753272 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:20.753543 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.754060 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.755079 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.755493 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.755567 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.755579 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.755769 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.755819 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:41:20.755888 32057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.752709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:35439 (local address 127.1.208.3:59647)
0808 00:41:20.752983 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.752993 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:20.753053 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:20.753357 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:20.753359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:20.753361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:20.753447 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.753448 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.754150 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.754151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.754973 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:20.754976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.755578 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.755579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:20.755653 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.755657 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:20.755751 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 00:41:20.755791 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":56,"thread_start_us":151,"threads_started":1}
I20260808 00:41:20.755965 31431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22784433ca84140836393da15de552a" candidate_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dac82dbac7467791932af21323572e" is_pre_election: true
I20260808 00:41:20.756165 31431 raft_consensus.cc:2454] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a30fe6b9f3d4c8a87513d454155a3da in term 0.
I20260808 00:41:20.756250 31690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22784433ca84140836393da15de552a" candidate_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a02eded021e47f694e898a078d1ce6e" is_pre_election: true
I20260808 00:41:20.756402 31690 raft_consensus.cc:2454] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2a30fe6b9f3d4c8a87513d454155a3da in term 0.
I20260808 00:41:20.756435 31809 leader_election.cc:302] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a30fe6b9f3d4c8a87513d454155a3da, b3dac82dbac7467791932af21323572e; no voters:
I20260808 00:41:20.756575 32056 raft_consensus.cc:2781] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:20.756632 32056 raft_consensus.cc:482] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:20.756660 32056 raft_consensus.cc:3037] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:20.757439 32056 raft_consensus.cc:504] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.757578 32056 leader_election.cc:288] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [CANDIDATE]: Term 1 election: Requested vote from peers 4a02eded021e47f694e898a078d1ce6e (127.1.208.2:35439), b3dac82dbac7467791932af21323572e (127.1.208.1:43573)
I20260808 00:41:20.757728 31690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22784433ca84140836393da15de552a" candidate_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a02eded021e47f694e898a078d1ce6e"
I20260808 00:41:20.757732 31431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b22784433ca84140836393da15de552a" candidate_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dac82dbac7467791932af21323572e"
I20260808 00:41:20.757823 31431 raft_consensus.cc:3037] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:20.757823 31690 raft_consensus.cc:3037] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:20.758754 31431 raft_consensus.cc:2454] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a30fe6b9f3d4c8a87513d454155a3da in term 1.
I20260808 00:41:20.758754 31690 raft_consensus.cc:2454] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2a30fe6b9f3d4c8a87513d454155a3da in term 1.
I20260808 00:41:20.758921 31809 leader_election.cc:302] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a30fe6b9f3d4c8a87513d454155a3da, b3dac82dbac7467791932af21323572e; no voters:
I20260808 00:41:20.759093 32056 raft_consensus.cc:2781] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:20.759279 32056 raft_consensus.cc:686] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 1 LEADER]: Becoming Leader. State: Replica: 2a30fe6b9f3d4c8a87513d454155a3da, State: Running, Role: LEADER
I20260808 00:41:20.759372 32056 consensus_queue.cc:237] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } }
I20260808 00:41:20.760334 31169 catalog_manager.cc:5495] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a30fe6b9f3d4c8a87513d454155a3da (127.1.208.3). New cstate: current_term: 1 leader_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a30fe6b9f3d4c8a87513d454155a3da" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 39649 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 } health_report { overall_health: UNKNOWN } } }
W20260808 00:41:20.801124 32020 tablet.cc:2369] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:20.802989 32039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.800480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59526 (local address 127.1.208.3:39649)
0808 00:41:20.800545 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:20.800547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:20.800561 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:41:20.800604 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:20.800605 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:20.800627 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:20.800688 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:20.800820 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.801461 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.802228 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:20.802432 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:20.802489 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:20.802646 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:20.802764 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:20.802836 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:20.802875 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 00:41:20.802897 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:41:20.804772 31431 raft_consensus.cc:1261] T b22784433ca84140836393da15de552a P b3dac82dbac7467791932af21323572e [term 1 FOLLOWER]: Refusing update from remote peer 2a30fe6b9f3d4c8a87513d454155a3da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:20.804773 31690 raft_consensus.cc:1261] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 1 FOLLOWER]: Refusing update from remote peer 2a30fe6b9f3d4c8a87513d454155a3da: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:20.805132 32056 consensus_queue.cc:1035] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dac82dbac7467791932af21323572e" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 43573 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:20.805303 32060 consensus_queue.cc:1035] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a02eded021e47f694e898a078d1ce6e" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 35439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:20.807761 32066 mvcc.cc:205] Tried to move back new op lower bound from 7316069092573618176 to 7316069092391092224. Current Snapshot: MvccSnapshot[applied={T|T < 7316069092573618176}]
I20260808 00:41:20.808202 32065 mvcc.cc:205] Tried to move back new op lower bound from 7316069092573618176 to 7316069092391092224. Current Snapshot: MvccSnapshot[applied={T|T < 7316069092573618176}]
I20260808 00:41:20.809933 31169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:20.810041 31169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:20.810819 31169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dff442e6601b452b8de138122a1dbcba: Sending DeleteTablet for 3 replicas of tablet b22784433ca84140836393da15de552a
I20260808 00:41:20.811178 31397 tablet_service.cc:1514] Processing DeleteTablet for tablet b22784433ca84140836393da15de552a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:20 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:57136
I20260808 00:41:20.811178 31893 tablet_service.cc:1514] Processing DeleteTablet for tablet b22784433ca84140836393da15de552a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:20 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59520
I20260808 00:41:20.811440 32082 tablet_replica.cc:330] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: stopping tablet replica
I20260808 00:41:20.811450 31639 tablet_service.cc:1514] Processing DeleteTablet for tablet b22784433ca84140836393da15de552a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:20 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:59666
I20260808 00:41:20.811542 32082 raft_consensus.cc:2229] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:41:20.811640 32082 raft_consensus.cc:2258] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:20.811820 32083 tablet_replica.cc:330] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: stopping tablet replica
I20260808 00:41:20.811921 32083 raft_consensus.cc:2229] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:20.812023 32083 raft_consensus.cc:2258] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:20.812079 32082 ts_tablet_manager.cc:1905] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:20.812414 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 31288
I20260808 00:41:20.815129 32083 ts_tablet_manager.cc:1905] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:20.817426 32083 ts_tablet_manager.cc:1918] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:20.817497 32083 log.cc:1196] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/b22784433ca84140836393da15de552a
I20260808 00:41:20.817767 32083 ts_tablet_manager.cc:1939] T b22784433ca84140836393da15de552a P 4a02eded021e47f694e898a078d1ce6e: Deleting consensus metadata
I20260808 00:41:20.820046 31148 catalog_manager.cc:4841] TS 4a02eded021e47f694e898a078d1ce6e (127.1.208.2:35439): tablet b22784433ca84140836393da15de552a (table test-table [id=28b8062ee91b4a069ba72b82dab4573d]) successfully deleted
I20260808 00:41:20.824991 32082 ts_tablet_manager.cc:1918] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:20.825090 32082 log.cc:1196] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/b22784433ca84140836393da15de552a
I20260808 00:41:20.825409 32082 ts_tablet_manager.cc:1939] T b22784433ca84140836393da15de552a P 2a30fe6b9f3d4c8a87513d454155a3da: Deleting consensus metadata
I20260808 00:41:20.825937 31143 catalog_manager.cc:4841] TS 2a30fe6b9f3d4c8a87513d454155a3da (127.1.208.3:39649): tablet b22784433ca84140836393da15de552a (table test-table [id=28b8062ee91b4a069ba72b82dab4573d]) successfully deleted
W20260808 00:41:20.827968 31143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:43573 (error 108)
I20260808 00:41:20.828167 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 31537
I20260808 00:41:20.838524 32034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:20.838383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:43573 (local address 127.0.0.1:57140)
0808 00:41:20.838440 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:20.838487 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:43573: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260808 00:41:20.838721 31143 catalog_manager.cc:4584] TS b3dac82dbac7467791932af21323572e (127.1.208.1:43573): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b22784433ca84140836393da15de552a: Network error: Client connection negotiation failed: client connection to 127.1.208.1:43573: connect: Connection refused (error 111)
I20260808 00:41:20.843501 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 31782
I20260808 00:41:20.858774 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 31114
2026-08-08T00:41:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2220 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T00:41:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:20Z Disabled control of system clock
I20260808 00:41:20.893414 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:44505
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:44505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:20.975114 32090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:20.975370 32090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:20.975402 32090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:20.976532 32090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:20.976573 32090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:20.976589 32090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:20.976605 32090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:20.976620 32090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:20.977982 32090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:44505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:44505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:20.978222 32090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:20.978489 32090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:20.983211 32109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:20.983162 32105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.983246 32090 server_base.cc:1034] running on GCE node
W20260808 00:41:20.983166 32103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:20.983664 32090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:20.983930 32090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:20.985088 32090 hybrid_clock.cc:648] HybridClock initialized: now 1786149680985061 us; error 45 us; skew 500 ppm
I20260808 00:41:20.986315 32090 webserver.cc:460] Webserver started at http://127.1.208.62:46849/ using document root <none> and password file <none>
I20260808 00:41:20.986526 32090 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:20.986574 32090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:20.986704 32090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:20.987813 32090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "3d5dca38844b434ab387b0744f1485ca"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1a6bc567cdac919f66d746b9d274d218"
server_key_iv: "0a024d051df559f494fb43f8a7f221c6"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.988149 32090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "3d5dca38844b434ab387b0744f1485ca"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1a6bc567cdac919f66d746b9d274d218"
server_key_iv: "0a024d051df559f494fb43f8a7f221c6"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.989459 32090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:20.990145 32118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:20.990301 32090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:20.990388 32090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "3d5dca38844b434ab387b0744f1485ca"
format_stamp: "Formatted at 2026-08-08 00:41:20 on dist-test-slave-gjw7"
server_key: "1a6bc567cdac919f66d746b9d274d218"
server_key_iv: "0a024d051df559f494fb43f8a7f221c6"
server_key_version: "encryptionkey@0"
I20260808 00:41:20.990448 32090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:20.997401 32090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:20.997669 32090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:20.997798 32090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:21.001668 32090 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:44505
I20260808 00:41:21.001713 32207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:44505 every 8 connection(s)
I20260808 00:41:21.002023 32090 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:21.003015 32208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:21.005396 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: Bootstrap starting.
I20260808 00:41:21.005949 32208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:21.006206 32208 log.cc:824] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:21.006844 32208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: No bootstrap required, opened a new log
I20260808 00:41:21.007790 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 32090
I20260808 00:41:21.007905 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:21.008092 1856 external_mini_cluster.cc:1428] Setting key 3041ef4de786bbb54cfd6c93f85ef832
I20260808 00:41:21.008613 32208 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } }
I20260808 00:41:21.008790 32208 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:21.008838 32208 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d5dca38844b434ab387b0744f1485ca, State: Initialized, Role: FOLLOWER
I20260808 00:41:21.008958 32208 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } }
I20260808 00:41:21.009034 32208 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:21.009069 32208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:21.009112 32208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:21.009812 32208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } }
I20260808 00:41:21.009941 32208 leader_election.cc:302] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d5dca38844b434ab387b0744f1485ca; no voters:
I20260808 00:41:21.010098 32208 leader_election.cc:288] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:21.010145 32215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:21.010347 32215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [term 1 LEADER]: Becoming Leader. State: Replica: 3d5dca38844b434ab387b0744f1485ca, State: Running, Role: LEADER
I20260808 00:41:21.010365 32208 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:21.010452 32215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } }
I20260808 00:41:21.010800 32215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d5dca38844b434ab387b0744f1485ca. Latest consensus state: current_term: 1 leader_uuid: "3d5dca38844b434ab387b0744f1485ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } } }
I20260808 00:41:21.010895 32215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:21.010810 32217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d5dca38844b434ab387b0744f1485ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d5dca38844b434ab387b0744f1485ca" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44505 } } }
I20260808 00:41:21.011060 32217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:21.011183 32223 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:21.011698 32223 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:21.013571 32223 catalog_manager.cc:1347] Generated new cluster ID: 0d186b296bfc46569823cbac4875e4bf
I20260808 00:41:21.013623 32223 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:21.014644 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.008605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.1.208.62:44505)
0808 00:41:21.008856 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.008866 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.008899 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:41:21.009013 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.009020 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.009091 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.009242 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:21.009499 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.011284 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.012424 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.013081 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.013206 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.013968 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:21.013972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:21.013973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:21.014144 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:21.014177 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:21.014187 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:21.014240 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.014326 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.014490 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":107,"threads_started":1}
I20260808 00:41:21.043249 32223 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:21.043767 32223 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:21.051575 32223 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: Generated new TSK 0
I20260808 00:41:21.051755 32223 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:21.057489 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--builtin_ntp_servers=127.1.208.20:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:21.158677 32248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:21.158880 32248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:21.158910 32248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:21.160025 32248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:21.160063 32248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:21.160096 32248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:21.161533 32248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:21.161765 32248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:21.162052 32248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:21.166978 32262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:21.166992 32265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:21.167060 32248 server_base.cc:1034] running on GCE node
W20260808 00:41:21.166968 32268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:21.167371 32248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:21.167604 32248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:21.168737 32248 hybrid_clock.cc:648] HybridClock initialized: now 1786149681168726 us; error 27 us; skew 500 ppm
I20260808 00:41:21.169992 32248 webserver.cc:460] Webserver started at http://127.1.208.1:42693/ using document root <none> and password file <none>
I20260808 00:41:21.170220 32248 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:21.170265 32248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:21.170387 32248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:21.171440 32248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "ee680e88daaf494d9828197cd310b291"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "93489091913c8ee466fa265b4b15606b"
server_key_iv: "758501b8ab56cec4f9581337b40e8950"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.171770 32248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "ee680e88daaf494d9828197cd310b291"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "93489091913c8ee466fa265b4b15606b"
server_key_iv: "758501b8ab56cec4f9581337b40e8950"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.173010 32248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:21.173754 32278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.173887 32248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.173971 32248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "ee680e88daaf494d9828197cd310b291"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "93489091913c8ee466fa265b4b15606b"
server_key_iv: "758501b8ab56cec4f9581337b40e8950"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.174050 32248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:21.191268 32248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:21.191541 32248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:21.191650 32248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:21.191877 32248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:21.192189 32248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:21.192222 32248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.192245 32248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:21.192267 32248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.197794 32248 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:37129
I20260808 00:41:21.197854 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:37129 every 8 connection(s)
I20260808 00:41:21.198154 32248 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:21.203244 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.198854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:32971 (local address 127.1.208.62:44505)
0808 00:41:21.198920 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.198922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.199169 (+ 247us) server_negotiation.cc:407] Connection header received
0808 00:41:21.199704 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.199705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.199722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.199776 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:21.200453 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.200954 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.201937 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.202134 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.202189 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.202987 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:21.202988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:21.202989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:21.202994 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:21.203009 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:21.203013 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:21.203055 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.203128 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.203163 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:41:21.203459 32502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.198870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44505 (local address 127.1.208.1:32971)
0808 00:41:21.199050 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.199084 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.199603 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.199862 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.199865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.199873 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:21.200366 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.200371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.201041 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.201042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.201855 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.201858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.202205 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.202206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.202332 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.202858 (+ 526us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:21.202858 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:21.202887 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:21.202909 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:21.203061 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:21.203067 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:21.203069 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.203127 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.203228 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260808 00:41:21.203945 32498 heartbeater.cc:347] Connected to a master server at 127.1.208.62:44505
I20260808 00:41:21.204049 32498 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:21.204115 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 32248
I20260808 00:41:21.204207 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:41:21.204258 32498 heartbeater.cc:511] Master 127.1.208.62:44505 requested a full tablet report, sending...
I20260808 00:41:21.204438 1856 external_mini_cluster.cc:1428] Setting key b962babbbb16a4ce4cd00c71613f4a41
I20260808 00:41:21.204808 32145 ts_manager.cc:194] Registered new tserver with Master: ee680e88daaf494d9828197cd310b291 (127.1.208.1:37129)
I20260808 00:41:21.205453 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--builtin_ntp_servers=127.1.208.20:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:41:21.205564 32145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.1:32971
W20260808 00:41:21.290759 32503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:21.290953 32503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:21.290973 32503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:21.292088 32503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:21.292129 32503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:21.292155 32503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:21.293591 32503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:21.293817 32503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:21.294099 32503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:21.298872 32512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:21.298964 32503 server_base.cc:1034] running on GCE node
W20260808 00:41:21.298864 32517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:21.298885 32520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:21.299285 32503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:21.299495 32503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:21.300652 32503 hybrid_clock.cc:648] HybridClock initialized: now 1786149681300636 us; error 28 us; skew 500 ppm
I20260808 00:41:21.301905 32503 webserver.cc:460] Webserver started at http://127.1.208.2:44515/ using document root <none> and password file <none>
I20260808 00:41:21.302131 32503 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:21.302177 32503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:21.302306 32503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:21.303339 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "d62556452c53f5ed1fdaeda8a250b03b"
server_key_iv: "696b55f7671baa1647cf1a5f2dbf6128"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.303660 32503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "d62556452c53f5ed1fdaeda8a250b03b"
server_key_iv: "696b55f7671baa1647cf1a5f2dbf6128"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.304841 32503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:21.305552 32527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.305702 32503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.305781 32503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "d62556452c53f5ed1fdaeda8a250b03b"
server_key_iv: "696b55f7671baa1647cf1a5f2dbf6128"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.305846 32503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:21.318347 32503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:21.318591 32503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:21.318713 32503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:21.318904 32503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:21.319204 32503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:21.319238 32503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.319259 32503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:21.319290 32503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.325109 32503 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:44253
I20260808 00:41:21.325165 32728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:44253 every 8 connection(s)
I20260808 00:41:21.325493 32503 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:41:21.330583 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.326157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:48523 (local address 127.1.208.62:44505)
0808 00:41:21.326225 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.326227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.326468 (+ 241us) server_negotiation.cc:407] Connection header received
0808 00:41:21.326984 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.326985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.326997 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.327047 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:21.327725 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.328239 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.329214 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.329457 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.329517 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.330327 (+ 810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:21.330328 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:21.330329 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:21.330334 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:21.330350 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:21.330356 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:21.330399 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.330473 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.330503 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:41:21.330801 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.326129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44505 (local address 127.1.208.2:48523)
0808 00:41:21.326355 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.326387 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.326884 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.327128 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.327131 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.327140 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:21.327633 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.327637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.328334 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.328336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.329121 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.329124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.329538 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.329539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.329669 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.330150 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:21.330150 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:21.330180 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:21.330245 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:21.330406 (+ 161us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:21.330411 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:21.330414 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.330473 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.330577 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":94,"threads_started":1}
I20260808 00:41:21.330850 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 32503
I20260808 00:41:21.330958 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:21.331174 1856 external_mini_cluster.cc:1428] Setting key fc0f7c6f0679dfc735f0c782887a9a11
I20260808 00:41:21.331300 32729 heartbeater.cc:347] Connected to a master server at 127.1.208.62:44505
I20260808 00:41:21.331396 32729 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:21.331585 32729 heartbeater.cc:511] Master 127.1.208.62:44505 requested a full tablet report, sending...
I20260808 00:41:21.331895 32145 ts_manager.cc:194] Registered new tserver with Master: 5246c7ee2dac41e6a5ad6fd68a6c5914 (127.1.208.2:44253)
I20260808 00:41:21.332131 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--builtin_ntp_servers=127.1.208.20:38517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 00:41:21.332363 32145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.2:48523
W20260808 00:41:21.414885 32734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:21.415103 32734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:21.415128 32734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:21.416277 32734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:21.416319 32734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:21.416342 32734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:21.417668 32734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:44505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:21.417881 32734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:21.418116 32734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:21.422876 32751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:21.422950 32747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:21.422871 32754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:21.422892 32734 server_base.cc:1034] running on GCE node
I20260808 00:41:21.423272 32734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:21.423537 32734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:21.424701 32734 hybrid_clock.cc:648] HybridClock initialized: now 1786149681424690 us; error 42 us; skew 500 ppm
I20260808 00:41:21.425890 32734 webserver.cc:460] Webserver started at http://127.1.208.3:37073/ using document root <none> and password file <none>
I20260808 00:41:21.426095 32734 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:21.426142 32734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:21.426272 32734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:21.427315 32734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "5089e8069a294c3885b91dffb73a7aeb"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "80ba6fb378a72351012b95d190cf08e4"
server_key_iv: "f0819e8ff9ec0ddb4c0b908077e93dbe"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.427649 32734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "5089e8069a294c3885b91dffb73a7aeb"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "80ba6fb378a72351012b95d190cf08e4"
server_key_iv: "f0819e8ff9ec0ddb4c0b908077e93dbe"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.428941 32734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:21.429639 32765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.429802 32734 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.429898 32734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "5089e8069a294c3885b91dffb73a7aeb"
format_stamp: "Formatted at 2026-08-08 00:41:21 on dist-test-slave-gjw7"
server_key: "80ba6fb378a72351012b95d190cf08e4"
server_key_iv: "f0819e8ff9ec0ddb4c0b908077e93dbe"
server_key_version: "encryptionkey@0"
I20260808 00:41:21.429960 32734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:21.478958 32734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:21.479283 32734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:21.479422 32734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:21.479681 32734 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:21.480044 32734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:21.480082 32734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.480113 32734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:21.480129 32734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:21.485782 32734 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:35671
I20260808 00:41:21.485829 495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:35671 every 8 connection(s)
I20260808 00:41:21.486163 32734 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:41:21.488358 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 32734
I20260808 00:41:21.488440 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:21.488637 1856 external_mini_cluster.cc:1428] Setting key aa904599528d097b2b01bffbbae522ce
I20260808 00:41:21.492519 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.486913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:58279 (local address 127.1.208.62:44505)
0808 00:41:21.486976 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.486978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.487217 (+ 239us) server_negotiation.cc:407] Connection header received
0808 00:41:21.487713 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.487714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.487732 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.487794 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:21.488481 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.489175 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.490893 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.491186 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.491281 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.492189 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:21.492191 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:21.492192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:21.492197 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:21.492219 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:21.492224 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:21.492289 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.492382 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.492421 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:41:21.492764 500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.486900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44505 (local address 127.1.208.3:58279)
0808 00:41:21.487100 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.487142 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.487631 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.487861 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.487864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.487873 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:21.488378 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.488383 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.489411 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.489593 (+ 182us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.490783 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.490787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.491288 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.491289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.491454 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.492029 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:21.492029 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:21.492069 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:21.492097 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:21.492295 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:21.492300 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:21.492303 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.492368 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.492484 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":78,"threads_started":1}
I20260808 00:41:21.493463 497 heartbeater.cc:347] Connected to a master server at 127.1.208.62:44505
I20260808 00:41:21.493580 497 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:21.493785 497 heartbeater.cc:511] Master 127.1.208.62:44505 requested a full tablet report, sending...
I20260808 00:41:21.494189 32145 ts_manager.cc:194] Registered new tserver with Master: 5089e8069a294c3885b91dffb73a7aeb (127.1.208.3:35671)
I20260808 00:41:21.494668 32145 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.3:58279
I20260808 00:41:21.500593 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:21.504575 32213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.502002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43306 (local address 127.1.208.62:44505)
0808 00:41:21.502044 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.502046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.502163 (+ 117us) server_negotiation.cc:407] Connection header received
0808 00:41:21.502223 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.502224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.502236 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.502287 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:21.502468 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.502917 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.503622 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.503804 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.503846 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.504422 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:21.504423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:21.504423 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:21.504426 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:21.504435 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:21.504439 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:21.504466 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.504503 (+ 37us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.504521 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:41:21.505990 32145 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:21.507515 32145 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:21.513980 508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.509679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37129 (local address 127.0.0.1:41026)
0808 00:41:21.509972 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.509982 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.510035 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.510561 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.510562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.510564 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:21.510614 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.510616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.511760 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.511761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.512754 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.512757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513667 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.513756 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.513771 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.513845 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.513901 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":3,"thread_start_us":73,"threads_started":1}
I20260808 00:41:21.514204 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.509886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.1.208.3:35671)
0808 00:41:21.510090 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.510096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.510105 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:21.510142 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.510144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.510158 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.510249 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:21.510439 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.511557 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.512998 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513720 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513815 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.513901 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.513952 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.513987 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":91,"threads_started":1}
I20260808 00:41:21.514340 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.509792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.1.208.1:37129)
0808 00:41:21.510378 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.510383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.510393 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:21.510432 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.510433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.510447 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.510671 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:21.510734 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.511669 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.512832 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513568 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513640 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.513731 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.514145 (+ 414us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.514188 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":91,"threads_started":1}
I20260808 00:41:21.514410 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.510100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44560 (local address 127.1.208.2:44253)
0808 00:41:21.510511 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.510516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.510526 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:21.510565 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.510567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.510582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.510654 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:21.510836 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.512052 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513199 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513811 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513868 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.513954 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.514291 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.514335 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":102,"threads_started":1}
I20260808 00:41:21.514077 511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.509693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:35671 (local address 127.0.0.1:57438)
0808 00:41:21.509818 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.509857 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.509914 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.510287 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.510290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.510292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:21.510362 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.510364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.511633 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.511634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.512915 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.512918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513817 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.513879 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.513883 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.513935 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.513975 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":32,"thread_start_us":55,"threads_started":1}
I20260808 00:41:21.514204 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.509911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:44253 (local address 127.0.0.1:44560)
0808 00:41:21.510095 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.510104 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.510148 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.510722 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.510724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.510726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:21.510766 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.510767 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.512121 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.512122 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.513128 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.513130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.513990 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.513991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.514050 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.514054 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.514105 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.514145 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":1,"thread_start_us":51,"threads_started":1}
I20260808 00:41:21.515215 32380 tablet_service.cc:1467] Processing CreateTablet for tablet 578975c4d8694b6088d991be2433acaa (DEFAULT_TABLE table=test-table [id=26352e0029e44aa890bdf83c826621e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:21.515394 371 tablet_service.cc:1467] Processing CreateTablet for tablet 578975c4d8694b6088d991be2433acaa (DEFAULT_TABLE table=test-table [id=26352e0029e44aa890bdf83c826621e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:21.515458 32616 tablet_service.cc:1467] Processing CreateTablet for tablet 578975c4d8694b6088d991be2433acaa (DEFAULT_TABLE table=test-table [id=26352e0029e44aa890bdf83c826621e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:21.515599 32380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578975c4d8694b6088d991be2433acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:21.515661 371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578975c4d8694b6088d991be2433acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:21.515755 32616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 578975c4d8694b6088d991be2433acaa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:21.518172 526 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Bootstrap starting.
I20260808 00:41:21.518810 526 tablet_bootstrap.cc:654] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:21.519065 526 log.cc:824] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:21.519122 527 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Bootstrap starting.
I20260808 00:41:21.519353 528 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: Bootstrap starting.
I20260808 00:41:21.519652 527 tablet_bootstrap.cc:654] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:21.519955 528 tablet_bootstrap.cc:654] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:21.519965 527 log.cc:824] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:21.520026 526 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: No bootstrap required, opened a new log
I20260808 00:41:21.520089 526 ts_tablet_manager.cc:1397] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:21.520241 528 log.cc:824] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:21.521018 528 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: No bootstrap required, opened a new log
I20260808 00:41:21.521100 528 ts_tablet_manager.cc:1397] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:21.521620 527 tablet_bootstrap.cc:492] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: No bootstrap required, opened a new log
I20260808 00:41:21.521680 527 ts_tablet_manager.cc:1397] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 00:41:21.522809 526 raft_consensus.cc:348] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.522984 526 raft_consensus.cc:374] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:21.523020 526 raft_consensus.cc:729] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5089e8069a294c3885b91dffb73a7aeb, State: Initialized, Role: FOLLOWER
I20260808 00:41:21.523155 526 consensus_queue.cc:260] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.523409 526 ts_tablet_manager.cc:1428] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260808 00:41:21.523456 497 heartbeater.cc:503] Master 127.1.208.62:44505 was elected leader, sending a full tablet report...
I20260808 00:41:21.523582 528 raft_consensus.cc:348] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.523579 527 raft_consensus.cc:348] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.523693 528 raft_consensus.cc:374] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:21.523721 528 raft_consensus.cc:729] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee680e88daaf494d9828197cd310b291, State: Initialized, Role: FOLLOWER
I20260808 00:41:21.523722 527 raft_consensus.cc:374] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:21.523757 527 raft_consensus.cc:729] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5246c7ee2dac41e6a5ad6fd68a6c5914, State: Initialized, Role: FOLLOWER
I20260808 00:41:21.523828 528 consensus_queue.cc:260] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.523854 527 consensus_queue.cc:260] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.524087 528 ts_tablet_manager.cc:1428] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:41:21.524096 527 ts_tablet_manager.cc:1428] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:41:21.524130 32498 heartbeater.cc:503] Master 127.1.208.62:44505 was elected leader, sending a full tablet report...
I20260808 00:41:21.524194 32729 heartbeater.cc:503] Master 127.1.208.62:44505 was elected leader, sending a full tablet report...
W20260808 00:41:21.576067 32730 tablet.cc:2369] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:21.626606 540 raft_consensus.cc:482] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:21.626731 540 raft_consensus.cc:504] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.627056 540 leader_election.cc:288] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee680e88daaf494d9828197cd310b291 (127.1.208.1:37129), 5089e8069a294c3885b91dffb73a7aeb (127.1.208.3:35671)
I20260808 00:41:21.630849 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.627328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:54797 (local address 127.1.208.1:37129)
0808 00:41:21.627377 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.627379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.627386 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:21.627414 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.627415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.627426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.627473 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:21.627697 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.628460 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.629950 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.630511 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.630579 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.630589 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.630733 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.630775 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:41:21.630849 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.627470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:42983 (local address 127.1.208.3:35671)
0808 00:41:21.627515 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.627516 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.627525 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:21.627555 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.627556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.627564 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.627606 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:21.627757 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.628468 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.629949 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.630504 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.630579 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.630589 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.630744 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.630782 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:41:21.630868 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.627221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37129 (local address 127.1.208.2:54797)
0808 00:41:21.627288 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.627299 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.627350 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.627527 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.627529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.627530 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:21.627606 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.627609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.628546 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.628547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.629861 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.629864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.630584 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.630585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.630664 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.630668 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.630720 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.630772 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":57}
I20260808 00:41:21.630877 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.627245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:35671 (local address 127.1.208.2:42983)
0808 00:41:21.627460 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.627468 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:21.627508 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:21.627625 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:21.627626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:21.627627 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:21.627683 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.627685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.628541 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.628541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.629861 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:21.629864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.630587 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.630587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:21.630669 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.630671 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:21.630720 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:41:21.630769 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":45,"thread_start_us":114,"threads_started":1}
I20260808 00:41:21.631278 32411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578975c4d8694b6088d991be2433acaa" candidate_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee680e88daaf494d9828197cd310b291" is_pre_election: true
I20260808 00:41:21.631458 32411 raft_consensus.cc:2454] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5246c7ee2dac41e6a5ad6fd68a6c5914 in term 0.
I20260808 00:41:21.631451 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578975c4d8694b6088d991be2433acaa" candidate_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5089e8069a294c3885b91dffb73a7aeb" is_pre_election: true
I20260808 00:41:21.631614 403 raft_consensus.cc:2454] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5246c7ee2dac41e6a5ad6fd68a6c5914 in term 0.
I20260808 00:41:21.631705 32531 leader_election.cc:302] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5246c7ee2dac41e6a5ad6fd68a6c5914, ee680e88daaf494d9828197cd310b291; no voters:
I20260808 00:41:21.631863 540 raft_consensus.cc:2781] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:21.631932 540 raft_consensus.cc:482] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:21.631948 540 raft_consensus.cc:3037] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:21.632683 540 raft_consensus.cc:504] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.632797 540 leader_election.cc:288] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [CANDIDATE]: Term 1 election: Requested vote from peers ee680e88daaf494d9828197cd310b291 (127.1.208.1:37129), 5089e8069a294c3885b91dffb73a7aeb (127.1.208.3:35671)
I20260808 00:41:21.632941 32411 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578975c4d8694b6088d991be2433acaa" candidate_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee680e88daaf494d9828197cd310b291"
I20260808 00:41:21.632973 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "578975c4d8694b6088d991be2433acaa" candidate_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5089e8069a294c3885b91dffb73a7aeb"
I20260808 00:41:21.633039 403 raft_consensus.cc:3037] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:21.633031 32411 raft_consensus.cc:3037] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:21.633978 32411 raft_consensus.cc:2454] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5246c7ee2dac41e6a5ad6fd68a6c5914 in term 1.
I20260808 00:41:21.633978 403 raft_consensus.cc:2454] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5246c7ee2dac41e6a5ad6fd68a6c5914 in term 1.
I20260808 00:41:21.634132 32531 leader_election.cc:302] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5246c7ee2dac41e6a5ad6fd68a6c5914, ee680e88daaf494d9828197cd310b291; no voters:
I20260808 00:41:21.634238 540 raft_consensus.cc:2781] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:21.634406 540 raft_consensus.cc:686] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 1 LEADER]: Becoming Leader. State: Replica: 5246c7ee2dac41e6a5ad6fd68a6c5914, State: Running, Role: LEADER
I20260808 00:41:21.634516 540 consensus_queue.cc:237] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } }
I20260808 00:41:21.635303 32145 catalog_manager.cc:5495] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5246c7ee2dac41e6a5ad6fd68a6c5914 (127.1.208.2). New cstate: current_term: 1 leader_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5246c7ee2dac41e6a5ad6fd68a6c5914" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 44253 } health_report { overall_health: HEALTHY } } }
I20260808 00:41:21.660444 525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.658393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44570 (local address 127.1.208.2:44253)
0808 00:41:21.658450 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:21.658452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:21.658459 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:21.658495 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:21.658496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:21.658515 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:21.658573 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:21.658671 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.659128 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.659799 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:21.659982 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:21.660027 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:21.660123 (+ 96us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:21.660248 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:21.660307 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:21.660320 (+ 13us) server_negotiation.cc:299] Negotiation successful
0808 00:41:21.660338 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 00:41:21.662334 403 raft_consensus.cc:1261] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 1 FOLLOWER]: Refusing update from remote peer 5246c7ee2dac41e6a5ad6fd68a6c5914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:21.662329 32411 raft_consensus.cc:1261] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 1 FOLLOWER]: Refusing update from remote peer 5246c7ee2dac41e6a5ad6fd68a6c5914: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:21.662681 540 consensus_queue.cc:1035] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee680e88daaf494d9828197cd310b291" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 37129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:21.662869 540 consensus_queue.cc:1035] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5089e8069a294c3885b91dffb73a7aeb" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 35671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:21.665397 551 mvcc.cc:205] Tried to move back new op lower bound from 7316069096086011904 to 7316069095975501824. Current Snapshot: MvccSnapshot[applied={T|T < 7316069096086011904}]
I20260808 00:41:21.665987 555 mvcc.cc:205] Tried to move back new op lower bound from 7316069096086011904 to 7316069095975501824. Current Snapshot: MvccSnapshot[applied={T|T < 7316069096086011904}]
I20260808 00:41:21.668396 32142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:21.668503 32142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:21.669368 32142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3d5dca38844b434ab387b0744f1485ca: Sending DeleteTablet for 3 replicas of tablet 578975c4d8694b6088d991be2433acaa
I20260808 00:41:21.669765 32380 tablet_service.cc:1514] Processing DeleteTablet for tablet 578975c4d8694b6088d991be2433acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:21 UTC) from {username='slave'} at 127.0.0.1:41026
I20260808 00:41:21.669787 371 tablet_service.cc:1514] Processing DeleteTablet for tablet 578975c4d8694b6088d991be2433acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:21 UTC) from {username='slave'} at 127.0.0.1:57438
I20260808 00:41:21.669831 32616 tablet_service.cc:1514] Processing DeleteTablet for tablet 578975c4d8694b6088d991be2433acaa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:21 UTC) from {username='slave'} at 127.0.0.1:44560
I20260808 00:41:21.670037 572 tablet_replica.cc:330] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: stopping tablet replica
I20260808 00:41:21.670089 571 tablet_replica.cc:330] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291: stopping tablet replica
I20260808 00:41:21.670204 572 raft_consensus.cc:2229] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:21.670186 571 raft_consensus.cc:2229] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:21.670246 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 32248
I20260808 00:41:21.670342 571 raft_consensus.cc:2258] T 578975c4d8694b6088d991be2433acaa P ee680e88daaf494d9828197cd310b291 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:21.682025 572 raft_consensus.cc:2258] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:21.682621 572 ts_tablet_manager.cc:1905] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:21.684077 574 tablet_replica.cc:330] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: stopping tablet replica
I20260808 00:41:21.684222 574 raft_consensus.cc:2229] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:41:21.684372 572 ts_tablet_manager.cc:1918] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:21.684465 572 log.cc:1196] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/578975c4d8694b6088d991be2433acaa
I20260808 00:41:21.684746 572 ts_tablet_manager.cc:1939] T 578975c4d8694b6088d991be2433acaa P 5089e8069a294c3885b91dffb73a7aeb: Deleting consensus metadata
I20260808 00:41:21.685333 32123 catalog_manager.cc:4841] TS 5089e8069a294c3885b91dffb73a7aeb (127.1.208.3:35671): tablet 578975c4d8694b6088d991be2433acaa (table test-table [id=26352e0029e44aa890bdf83c826621e6]) successfully deleted
I20260808 00:41:21.685683 574 raft_consensus.cc:2258] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:21.686249 574 ts_tablet_manager.cc:1905] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:21.687940 574 ts_tablet_manager.cc:1918] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:21.688014 574 log.cc:1196] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/578975c4d8694b6088d991be2433acaa
I20260808 00:41:21.688242 574 ts_tablet_manager.cc:1939] T 578975c4d8694b6088d991be2433acaa P 5246c7ee2dac41e6a5ad6fd68a6c5914: Deleting consensus metadata
I20260808 00:41:21.688760 32123 catalog_manager.cc:4841] TS 5246c7ee2dac41e6a5ad6fd68a6c5914 (127.1.208.2:44253): tablet 578975c4d8694b6088d991be2433acaa (table test-table [id=26352e0029e44aa890bdf83c826621e6]) successfully deleted
W20260808 00:41:21.690865 32120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:37129 (error 108)
I20260808 00:41:21.691247 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 32503
I20260808 00:41:21.691349 518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:21.691212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:37129 (local address 127.0.0.1:41030)
0808 00:41:21.691266 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:21.691310 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:37129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260808 00:41:21.693589 32120 catalog_manager.cc:4584] TS ee680e88daaf494d9828197cd310b291 (127.1.208.1:37129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 578975c4d8694b6088d991be2433acaa: Network error: Client connection negotiation failed: client connection to 127.1.208.1:37129: connect: Connection refused (error 111)
I20260808 00:41:21.701530 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 32734
I20260808 00:41:21.726716 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 32090
2026-08-08T00:41:21Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (871 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:21 dist-test-slave-gjw7 krb5kdc[579](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:21 dist-test-slave-gjw7 krb5kdc[579](info): set up 2 sockets
Aug 08 00:41:21 dist-test-slave-gjw7 krb5kdc[579](info): commencing operation
krb5kdc: starting...
W20260808 00:41:22.900782 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:22 dist-test-slave-gjw7 krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:23.018013 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34559
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34559
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:23.098255 603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:23.098448 603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:23.098469 603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:23.099527 603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:23.099560 603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:23.099571 603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:23.099582 603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:23.099592 603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:23.100973 603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:23.101155 603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:23.101337 603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:23.105883 613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.105897 610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.105897 616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:23.106195 603 server_base.cc:1034] running on GCE node
I20260808 00:41:23.106416 603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:23.106667 603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:23.107800 603 hybrid_clock.cc:648] HybridClock initialized: now 1786149683107784 us; error 32 us; skew 500 ppm
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:23.110219 603 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:23.110652 603 webserver.cc:460] Webserver started at http://127.1.208.62:37539/ using document root <none> and password file <none>
I20260808 00:41:23.110853 603 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:23.110903 603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:23.111027 603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:23.112066 603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "b972e424a8f84af4a7938f74a05e00d5"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b6fd9b66a1ece61455a05d1806982698"
server_key_iv: "941ec7b3920838787508fc74f9077b56"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.112396 603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "b972e424a8f84af4a7938f74a05e00d5"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b6fd9b66a1ece61455a05d1806982698"
server_key_iv: "941ec7b3920838787508fc74f9077b56"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.113629 603 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260808 00:41:23.114260 627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.114403 603 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.114485 603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "b972e424a8f84af4a7938f74a05e00d5"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b6fd9b66a1ece61455a05d1806982698"
server_key_iv: "941ec7b3920838787508fc74f9077b56"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.114543 603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:23.129555 603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:23.131554 603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:23.131686 603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:23.135635 603 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34559
I20260808 00:41:23.135689 736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34559 every 8 connection(s)
I20260808 00:41:23.136010 603 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:23.136886 737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:23.139154 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: Bootstrap starting.
I20260808 00:41:23.139711 737 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:23.139981 737 log.cc:824] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:23.140600 737 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: No bootstrap required, opened a new log
I20260808 00:41:23.142231 737 raft_consensus.cc:348] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } }
I20260808 00:41:23.142369 737 raft_consensus.cc:374] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:23.142402 737 raft_consensus.cc:729] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b972e424a8f84af4a7938f74a05e00d5, State: Initialized, Role: FOLLOWER
I20260808 00:41:23.142454 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 603
I20260808 00:41:23.142551 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:23.142534 737 consensus_queue.cc:260] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } }
I20260808 00:41:23.142614 737 raft_consensus.cc:388] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:23.142653 737 raft_consensus.cc:482] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:23.142701 737 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:23.142725 1856 external_mini_cluster.cc:1428] Setting key 9cd7b14c8bc6cc3e7f8a77322cb20cb2
I20260808 00:41:23.143326 737 raft_consensus.cc:504] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } }
I20260808 00:41:23.143424 737 leader_election.cc:302] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b972e424a8f84af4a7938f74a05e00d5; no voters:
I20260808 00:41:23.143594 737 leader_election.cc:288] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:23.143641 744 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:23.143778 744 raft_consensus.cc:686] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [term 1 LEADER]: Becoming Leader. State: Replica: b972e424a8f84af4a7938f74a05e00d5, State: Running, Role: LEADER
I20260808 00:41:23.143854 737 sys_catalog.cc:564] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:23.143867 744 consensus_queue.cc:237] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } }
I20260808 00:41:23.144291 744 sys_catalog.cc:455] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b972e424a8f84af4a7938f74a05e00d5. Latest consensus state: current_term: 1 leader_uuid: "b972e424a8f84af4a7938f74a05e00d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } } }
I20260808 00:41:23.144366 744 sys_catalog.cc:458] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:23.144590 756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:23.145126 756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:23.145367 746 sys_catalog.cc:455] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b972e424a8f84af4a7938f74a05e00d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b972e424a8f84af4a7938f74a05e00d5" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34559 } } }
I20260808 00:41:23.146385 746 sys_catalog.cc:458] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:23.146677 756 catalog_manager.cc:1347] Generated new cluster ID: 49df37ac9d9e48bebb4c5f4d02337b0b
I20260808 00:41:23.146726 756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149682, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:23.154865 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.143468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60208 (local address 127.1.208.62:34559)
0808 00:41:23.143660 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.143670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.143708 (+ 38us) server_negotiation.cc:407] Connection header received
0808 00:41:23.143825 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.143831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.143885 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.144024 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:23.144905 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.146352 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.147426 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.148033 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.148110 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.150909 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:23.150913 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:23.150914 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:23.150925 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:23.153318 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.153697 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.153699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.153700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.153794 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.153979 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.153980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.153980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.154252 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:23.154380 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.154483 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.154607 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":90,"threads_started":1}
I20260808 00:41:23.168958 756 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:23.169528 756 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:23.178053 756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: Generated new TSK 0
I20260808 00:41:23.178237 756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:41:23.209987 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:23.292958 781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:23.293181 781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:23.293216 781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:23.294377 781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:23.294417 781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:23.294452 781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:23.295938 781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:23.296170 781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:23.296382 781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:23.301435 793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:23.301538 781 server_base.cc:1034] running on GCE node
W20260808 00:41:23.301446 789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.301442 790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:23.301810 781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:23.302033 781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:23.303166 781 hybrid_clock.cc:648] HybridClock initialized: now 1786149683303152 us; error 25 us; skew 500 ppm
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:23.305581 781 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:41:23.306027 781 webserver.cc:460] Webserver started at http://127.1.208.1:46255/ using document root <none> and password file <none>
I20260808 00:41:23.306248 781 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:23.306298 781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:23.306420 781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:23.307448 781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "dcf1098d17e843dbb49cd767588538d3"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b5d19ddeeb43d94564ea5be972a508f7"
server_key_iv: "289ae66ea23b0b97d1f85c6f390e698e"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.307794 781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "dcf1098d17e843dbb49cd767588538d3"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b5d19ddeeb43d94564ea5be972a508f7"
server_key_iv: "289ae66ea23b0b97d1f85c6f390e698e"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.309010 781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:23.309731 804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.309877 781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.309952 781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "dcf1098d17e843dbb49cd767588538d3"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "b5d19ddeeb43d94564ea5be972a508f7"
server_key_iv: "289ae66ea23b0b97d1f85c6f390e698e"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.310020 781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:23.321386 781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:23.323021 781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:23.323150 781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:23.323357 781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:23.323640 781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:23.323674 781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.323701 781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:23.323719 781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.329340 781 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:33359
I20260808 00:41:23.329396 1004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:33359 every 8 connection(s)
I20260808 00:41:23.329674 781 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:23.335276 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 781
I20260808 00:41:23.335391 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:23.335610 1856 external_mini_cluster.cc:1428] Setting key 9ffbb7f4c169f36f4ec071c3588f22dd
I20260808 00:41:23.338722 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.330356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:36015 (local address 127.1.208.62:34559)
0808 00:41:23.330405 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.330407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.330600 (+ 193us) server_negotiation.cc:407] Connection header received
0808 00:41:23.331080 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.331081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.331092 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.331143 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:23.332021 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.332561 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.333642 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.333842 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.333890 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.336043 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:23.336044 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:23.336044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:23.336051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:23.337360 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.337798 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.337799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.337799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.337856 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.338113 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.338114 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.338114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.338249 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:23.338312 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.338545 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.338633 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260808 00:41:23.338802 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.330322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34559 (local address 127.1.208.1:36015)
0808 00:41:23.330476 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.330525 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.331005 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.331200 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.331203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.331761 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:23.331953 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.331959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.332635 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.332636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.333580 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.333583 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.333900 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.333901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.333995 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.334413 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:23.334421 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:23.335968 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:23.337430 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.337439 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.337441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.337731 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.337908 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.337908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.337909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.338047 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.338306 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:23.338311 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:23.338427 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.338509 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.338628 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260808 00:41:23.339268 1006 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34559
I20260808 00:41:23.339355 1006 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:23.339519 1006 heartbeater.cc:511] Master 127.1.208.62:34559 requested a full tablet report, sending...
I20260808 00:41:23.340060 672 ts_manager.cc:194] Registered new tserver with Master: dcf1098d17e843dbb49cd767588538d3 (127.1.208.1:33359)
I20260808 00:41:23.340898 672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:36015
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:41:23.361954 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:23.443790 1013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:23.444018 1013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:23.444046 1013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:23.445160 1013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:23.445197 1013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:23.445222 1013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:23.446648 1013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:23.446838 1013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:23.447019 1013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:23.451743 1026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.451785 1022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.451789 1024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:23.452050 1013 server_base.cc:1034] running on GCE node
I20260808 00:41:23.452252 1013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:23.452461 1013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:23.453601 1013 hybrid_clock.cc:648] HybridClock initialized: now 1786149683453582 us; error 29 us; skew 500 ppm
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:23.456069 1013 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:41:23.456538 1013 webserver.cc:460] Webserver started at http://127.1.208.2:44055/ using document root <none> and password file <none>
I20260808 00:41:23.456766 1013 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:23.456823 1013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:23.456943 1013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:23.458002 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "59596a39477ec6ef72f1556d503ef7d3"
server_key_iv: "d7fa285c6541c63df0ec10569b847a5f"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.458323 1013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "59596a39477ec6ef72f1556d503ef7d3"
server_key_iv: "d7fa285c6541c63df0ec10569b847a5f"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.459556 1013 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:23.460242 1043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.460392 1013 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:23.460470 1013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "59596a39477ec6ef72f1556d503ef7d3"
server_key_iv: "d7fa285c6541c63df0ec10569b847a5f"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.460539 1013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:23.477632 1013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:23.479729 1013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:23.479866 1013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:23.480089 1013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:23.480396 1013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:23.480429 1013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.480451 1013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:23.480474 1013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.486007 1013 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:42575
I20260808 00:41:23.486104 1262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:42575 every 8 connection(s)
I20260808 00:41:23.486357 1013 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:41:23.486924 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1013
I20260808 00:41:23.487030 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:23.487243 1856 external_mini_cluster.cc:1428] Setting key 737340136d54ecc558db7f477a14ddf9
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:23.496253 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.487677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:41853 (local address 127.1.208.62:34559)
0808 00:41:23.487719 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.487721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.488006 (+ 285us) server_negotiation.cc:407] Connection header received
0808 00:41:23.488511 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.488512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.488523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.488574 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:23.489552 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.490057 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.491037 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.491238 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.491294 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.493561 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:23.493562 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:23.493562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:23.493569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:23.494808 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.495306 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.495306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.495307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.495355 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.495624 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.495624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.495625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.495721 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:23.495766 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.496114 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.496182 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:23.496513 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.487693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34559 (local address 127.1.208.2:41853)
0808 00:41:23.487871 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.487917 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.488418 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.488650 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.488654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.489206 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:23.489467 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.489473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.490140 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.490141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.490954 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.490957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.491310 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.491311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.491449 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.491939 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:23.491949 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:23.493463 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:23.494897 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.494905 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.494907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.495221 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.495415 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.495416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.495416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.495559 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.495780 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:23.495786 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:23.495970 (+ 184us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.496096 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.496276 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
I20260808 00:41:23.497035 1263 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34559
I20260808 00:41:23.497133 1263 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:23.497344 1263 heartbeater.cc:511] Master 127.1.208.62:34559 requested a full tablet report, sending...
I20260808 00:41:23.497751 672 ts_manager.cc:194] Registered new tserver with Master: 5af1ca69dbe74a1188a7b48e8ce2e8bd (127.1.208.2:42575)
I20260808 00:41:23.498226 672 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:41853
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:41:23.514148 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:23.597544 1271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:23.597739 1271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:23.597761 1271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:23.598820 1271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:23.598852 1271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:23.598875 1271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:23.600291 1271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:23.600469 1271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:23.600648 1271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:23.605347 1286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.605379 1290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:23.605396 1284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:23.605530 1271 server_base.cc:1034] running on GCE node
I20260808 00:41:23.605785 1271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:23.605983 1271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:23.607118 1271 hybrid_clock.cc:648] HybridClock initialized: now 1786149683607104 us; error 30 us; skew 500 ppm
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:23.609531 1271 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:41:23.609984 1271 webserver.cc:460] Webserver started at http://127.1.208.3:41085/ using document root <none> and password file <none>
I20260808 00:41:23.610198 1271 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:23.610247 1271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:23.610365 1271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:23.611363 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "ff4eaeab2912444d9c01f22bac3bc313"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "6dd227b8dd220b8dda11188df3e790b6"
server_key_iv: "af4121dd272ee2373e9ecafbd6da8198"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.611680 1271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "ff4eaeab2912444d9c01f22bac3bc313"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "6dd227b8dd220b8dda11188df3e790b6"
server_key_iv: "af4121dd272ee2373e9ecafbd6da8198"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.612886 1271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:23.613552 1302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.613679 1271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:23.613766 1271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "ff4eaeab2912444d9c01f22bac3bc313"
format_stamp: "Formatted at 2026-08-08 00:41:23 on dist-test-slave-gjw7"
server_key: "6dd227b8dd220b8dda11188df3e790b6"
server_key_iv: "af4121dd272ee2373e9ecafbd6da8198"
server_key_version: "encryptionkey@0"
I20260808 00:41:23.613826 1271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:23.622524 1271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:23.624121 1271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:23.624248 1271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:23.624442 1271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:23.624742 1271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:23.624779 1271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.624809 1271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:23.624832 1271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:23.630529 1271 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:33385
I20260808 00:41:23.630566 1494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:33385 every 8 connection(s)
I20260808 00:41:23.630868 1271 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:41:23 dist-test-slave-gjw7 krb5kdc[579](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149683, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:23.639343 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.631549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:57193 (local address 127.1.208.62:34559)
0808 00:41:23.631615 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.631617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.631795 (+ 178us) server_negotiation.cc:407] Connection header received
0808 00:41:23.632297 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.632298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.632312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.632363 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:23.633325 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.633849 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.634892 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.635108 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.635164 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.637135 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:23.637137 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:23.637137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:23.637144 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:23.638285 (+ 1141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.638626 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.638627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.638627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.638677 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.638868 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.638868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.638868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.638963 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:23.639006 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.639208 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.639274 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:41:23.639499 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.631503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:34559 (local address 127.1.208.3:57193)
0808 00:41:23.631677 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.631716 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.632195 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.632432 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.632436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.632997 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:23.633183 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.633188 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.633937 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.633938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.634807 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.634810 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.635181 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.635182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.635305 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.635706 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:23.635716 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:23.637041 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:23.638355 (+ 1314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.638361 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.638363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.638570 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.638737 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:23.638737 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:23.638738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:23.638825 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:23.639002 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:23.639005 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:23.639111 (+ 106us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.639185 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.639303 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
I20260808 00:41:23.639745 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1271
I20260808 00:41:23.639837 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:23.640120 1856 external_mini_cluster.cc:1428] Setting key 47f80d92f70821a7f03b32a7d9cdba9c
I20260808 00:41:23.640187 1495 heartbeater.cc:347] Connected to a master server at 127.1.208.62:34559
I20260808 00:41:23.640290 1495 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:23.640566 1495 heartbeater.cc:511] Master 127.1.208.62:34559 requested a full tablet report, sending...
I20260808 00:41:23.640974 669 ts_manager.cc:194] Registered new tserver with Master: ff4eaeab2912444d9c01f22bac3bc313 (127.1.208.3:33385)
I20260808 00:41:23.641477 669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:57193
I20260808 00:41:23.651914 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:23.659257 742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.653265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60216 (local address 127.1.208.62:34559)
0808 00:41:23.653348 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.653350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.653445 (+ 95us) server_negotiation.cc:407] Connection header received
0808 00:41:23.653490 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.653491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.653500 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.653545 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:23.654090 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.654554 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.655331 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.655503 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.655545 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.656932 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:23.656934 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:23.656934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:23.656940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:23.658168 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.658533 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.658534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.658534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.658583 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:23.658806 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:23.658807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:23.658807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:23.658929 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:23.659003 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.659129 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.659177 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 00:41:23.660936 669 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:23.662575 669 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:23.668777 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.1.208.1:33359)
0808 00:41:23.665154 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.665157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.665167 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:23.665207 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.665209 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.665225 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.665376 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:23.665662 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666756 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.667895 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668401 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668464 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668527 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.668652 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.668695 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":93,"threads_started":1}
I20260808 00:41:23.668772 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:33359 (local address 127.0.0.1:41000)
0808 00:41:23.664997 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.665006 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.665041 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.665481 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.665485 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.665489 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:23.665575 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.665577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666880 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.666881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.667802 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.667805 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668478 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.668556 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668571 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.668630 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.668683 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":45,"thread_start_us":60,"threads_started":1}
I20260808 00:41:23.668793 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:42575 (local address 127.0.0.1:60590)
0808 00:41:23.664796 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.664833 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.664897 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.665565 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.665566 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.665568 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:23.665612 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.665614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666467 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.666468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.667621 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.667624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668562 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.668622 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668626 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.668669 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.668709 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"spinlock_wait_cycles":896,"thread_start_us":62,"threads_started":1}
I20260808 00:41:23.668962 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:33385 (local address 127.0.0.1:35174)
0808 00:41:23.665140 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.665147 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.665190 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.665482 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.665485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.665487 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:23.665575 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.665578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666873 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.666874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.667878 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.667881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668741 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.668807 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668811 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.668869 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.668903 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":26,"thread_start_us":52,"threads_started":1}
I20260808 00:41:23.668967 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60590 (local address 127.1.208.2:42575)
0808 00:41:23.665322 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.665328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.665338 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:23.665381 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.665383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.665398 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.665475 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:23.665687 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666375 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.667704 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668405 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668472 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668526 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.668832 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.668882 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":94,"threads_started":1}
I20260808 00:41:23.669219 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.664913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35174 (local address 127.1.208.3:33385)
0808 00:41:23.665098 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.665104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.665200 (+ 96us) server_negotiation.cc:407] Connection header received
0808 00:41:23.665259 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.665261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.665275 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.665384 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:23.665662 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.666778 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.667941 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.668601 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.668659 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.668709 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.669121 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.669158 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260808 00:41:23.669734 1133 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623bd57a224ff784302af22e43806e (DEFAULT_TABLE table=test-table [id=bbb2341dd9ff4c4281c85c911ee1c550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:23.669910 1380 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623bd57a224ff784302af22e43806e (DEFAULT_TABLE table=test-table [id=bbb2341dd9ff4c4281c85c911ee1c550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:23.669955 887 tablet_service.cc:1467] Processing CreateTablet for tablet 8c623bd57a224ff784302af22e43806e (DEFAULT_TABLE table=test-table [id=bbb2341dd9ff4c4281c85c911ee1c550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:23.670121 1133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623bd57a224ff784302af22e43806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:23.670178 1380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623bd57a224ff784302af22e43806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:23.670282 887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c623bd57a224ff784302af22e43806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:23.673429 1520 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Bootstrap starting.
I20260808 00:41:23.673429 1519 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Bootstrap starting.
I20260808 00:41:23.673642 1517 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Bootstrap starting.
I20260808 00:41:23.674089 1520 tablet_bootstrap.cc:654] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:23.674089 1519 tablet_bootstrap.cc:654] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:23.674212 1517 tablet_bootstrap.cc:654] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:23.674669 1520 log.cc:824] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:23.674669 1519 log.cc:824] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:23.674742 1517 log.cc:824] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:23.675436 1520 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: No bootstrap required, opened a new log
I20260808 00:41:23.675462 1517 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: No bootstrap required, opened a new log
I20260808 00:41:23.675501 1520 ts_tablet_manager.cc:1397] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:41:23.675522 1517 ts_tablet_manager.cc:1397] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:23.675634 1519 tablet_bootstrap.cc:492] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: No bootstrap required, opened a new log
I20260808 00:41:23.675695 1519 ts_tablet_manager.cc:1397] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:41:23.677023 1520 raft_consensus.cc:348] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.677162 1520 raft_consensus.cc:374] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:23.677196 1520 raft_consensus.cc:729] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf1098d17e843dbb49cd767588538d3, State: Initialized, Role: FOLLOWER
I20260808 00:41:23.677341 1520 consensus_queue.cc:260] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.677436 1517 raft_consensus.cc:348] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.677614 1517 raft_consensus.cc:374] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:23.677641 1520 ts_tablet_manager.cc:1428] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:41:23.677655 1517 raft_consensus.cc:729] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff4eaeab2912444d9c01f22bac3bc313, State: Initialized, Role: FOLLOWER
I20260808 00:41:23.677567 1519 raft_consensus.cc:348] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.677731 1519 raft_consensus.cc:374] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:23.677771 1519 raft_consensus.cc:729] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af1ca69dbe74a1188a7b48e8ce2e8bd, State: Initialized, Role: FOLLOWER
I20260808 00:41:23.677798 1006 heartbeater.cc:503] Master 127.1.208.62:34559 was elected leader, sending a full tablet report...
I20260808 00:41:23.677781 1517 consensus_queue.cc:260] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.677877 1519 consensus_queue.cc:260] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.678045 1517 ts_tablet_manager.cc:1428] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 00:41:23.678125 1519 ts_tablet_manager.cc:1428] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 00:41:23.678174 1495 heartbeater.cc:503] Master 127.1.208.62:34559 was elected leader, sending a full tablet report...
I20260808 00:41:23.678234 1263 heartbeater.cc:503] Master 127.1.208.62:34559 was elected leader, sending a full tablet report...
W20260808 00:41:23.736944 1264 tablet.cc:2369] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:41:23.830391 1007 tablet.cc:2369] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:41:23.881460 1496 tablet.cc:2369] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:23.969198 1533 raft_consensus.cc:482] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:23.969389 1533 raft_consensus.cc:504] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.969775 1533 leader_election.cc:288] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dcf1098d17e843dbb49cd767588538d3 (127.1.208.1:33359), ff4eaeab2912444d9c01f22bac3bc313 (127.1.208.3:33385)
I20260808 00:41:23.973647 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.970138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:59333 (local address 127.1.208.1:33359)
0808 00:41:23.970213 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.970215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.970229 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:41:23.970255 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.970256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.970268 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.970336 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:23.970500 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.971252 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.972710 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.973267 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.973366 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.973378 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.973536 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.973572 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 00:41:23.973647 1515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.970252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:36939 (local address 127.1.208.3:33385)
0808 00:41:23.970314 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:23.970315 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:23.970324 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:23.970350 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:23.970351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:23.970363 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:23.970411 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:23.970550 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.971253 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.972711 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.973284 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.973366 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.973378 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:23.973536 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 00:41:23.973574 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 00:41:23.973680 1266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.969973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:33359 (local address 127.1.208.2:59333)
0808 00:41:23.970053 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.970066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.970119 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.970344 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.970345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.970347 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:23.970428 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.970430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.971334 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.971334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.972629 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.972632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.973373 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.973374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.973469 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.973474 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.973525 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.973572 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":75}
I20260808 00:41:23.973685 1536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:23.969983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:33385 (local address 127.1.208.2:36939)
0808 00:41:23.970103 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:23.970112 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:23.970156 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:23.970431 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:23.970432 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:23.970434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:23.970490 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.970491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.971332 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.971333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.972629 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:23.972632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:23.973374 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:23.973375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:23.973469 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:23.973474 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:23.973525 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:41:23.973571 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":42,"thread_start_us":77,"threads_started":1}
I20260808 00:41:23.974083 925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623bd57a224ff784302af22e43806e" candidate_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf1098d17e843dbb49cd767588538d3" is_pre_election: true
I20260808 00:41:23.974206 925 raft_consensus.cc:2454] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af1ca69dbe74a1188a7b48e8ce2e8bd in term 0.
I20260808 00:41:23.974220 1420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623bd57a224ff784302af22e43806e" candidate_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff4eaeab2912444d9c01f22bac3bc313" is_pre_election: true
I20260808 00:41:23.974354 1420 raft_consensus.cc:2454] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af1ca69dbe74a1188a7b48e8ce2e8bd in term 0.
I20260808 00:41:23.974442 1045 leader_election.cc:302] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af1ca69dbe74a1188a7b48e8ce2e8bd, dcf1098d17e843dbb49cd767588538d3; no voters:
I20260808 00:41:23.974562 1533 raft_consensus.cc:2781] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:23.974622 1533 raft_consensus.cc:482] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:23.974643 1533 raft_consensus.cc:3037] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:23.975401 1533 raft_consensus.cc:504] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.975520 1533 leader_election.cc:288] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [CANDIDATE]: Term 1 election: Requested vote from peers dcf1098d17e843dbb49cd767588538d3 (127.1.208.1:33359), ff4eaeab2912444d9c01f22bac3bc313 (127.1.208.3:33385)
I20260808 00:41:23.975673 1420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623bd57a224ff784302af22e43806e" candidate_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff4eaeab2912444d9c01f22bac3bc313"
I20260808 00:41:23.975687 925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c623bd57a224ff784302af22e43806e" candidate_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf1098d17e843dbb49cd767588538d3"
I20260808 00:41:23.975750 925 raft_consensus.cc:3037] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:23.975747 1420 raft_consensus.cc:3037] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:23.976462 925 raft_consensus.cc:2454] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af1ca69dbe74a1188a7b48e8ce2e8bd in term 1.
I20260808 00:41:23.976480 1420 raft_consensus.cc:2454] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af1ca69dbe74a1188a7b48e8ce2e8bd in term 1.
I20260808 00:41:23.976627 1044 leader_election.cc:302] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5af1ca69dbe74a1188a7b48e8ce2e8bd, ff4eaeab2912444d9c01f22bac3bc313; no voters:
I20260808 00:41:23.976727 1533 raft_consensus.cc:2781] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:23.976877 1533 raft_consensus.cc:686] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 1 LEADER]: Becoming Leader. State: Replica: 5af1ca69dbe74a1188a7b48e8ce2e8bd, State: Running, Role: LEADER
I20260808 00:41:23.976995 1533 consensus_queue.cc:237] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } }
I20260808 00:41:23.977793 669 catalog_manager.cc:5495] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af1ca69dbe74a1188a7b48e8ce2e8bd (127.1.208.2). New cstate: current_term: 1 leader_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af1ca69dbe74a1188a7b48e8ce2e8bd" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 42575 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 } health_report { overall_health: UNKNOWN } } }
I20260808 00:41:24.019423 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.016937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60594 (local address 127.1.208.2:42575)
0808 00:41:24.017000 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.017002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.017011 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:24.017057 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.017058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.017081 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.017140 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:24.017328 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.017842 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.018583 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.018792 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.018848 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.019010 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:24.019132 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:24.019219 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.019313 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.019321 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:41:24.021226 925 raft_consensus.cc:1261] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 1 FOLLOWER]: Refusing update from remote peer 5af1ca69dbe74a1188a7b48e8ce2e8bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:24.021255 1420 raft_consensus.cc:1261] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 1 FOLLOWER]: Refusing update from remote peer 5af1ca69dbe74a1188a7b48e8ce2e8bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:24.021656 1537 consensus_queue.cc:1035] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcf1098d17e843dbb49cd767588538d3" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:24.021788 1533 consensus_queue.cc:1035] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff4eaeab2912444d9c01f22bac3bc313" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:24.024926 1542 mvcc.cc:205] Tried to move back new op lower bound from 7316069105748201472 to 7316069105570357248. Current Snapshot: MvccSnapshot[applied={T|T < 7316069105748201472 or (T in {7316069105748201472})}]
I20260808 00:41:24.025063 1544 mvcc.cc:205] Tried to move back new op lower bound from 7316069105748201472 to 7316069105570357248. Current Snapshot: MvccSnapshot[applied={T|T < 7316069105748201472}]
I20260808 00:41:24.026407 672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:24.026489 672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60216:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:24.027207 672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b972e424a8f84af4a7938f74a05e00d5: Sending DeleteTablet for 3 replicas of tablet 8c623bd57a224ff784302af22e43806e
I20260808 00:41:24.027572 1380 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c623bd57a224ff784302af22e43806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:35174
I20260808 00:41:24.027577 1133 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c623bd57a224ff784302af22e43806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:60590
I20260808 00:41:24.027716 887 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c623bd57a224ff784302af22e43806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:41000
I20260808 00:41:24.027814 1564 tablet_replica.cc:330] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: stopping tablet replica
I20260808 00:41:24.027904 1564 raft_consensus.cc:2229] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:24.027946 1566 tablet_replica.cc:330] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: stopping tablet replica
I20260808 00:41:24.028000 1564 raft_consensus.cc:2258] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:24.028044 1566 raft_consensus.cc:2229] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:24.028151 1566 raft_consensus.cc:2258] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:24.028388 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 781
I20260808 00:41:24.028472 1564 ts_tablet_manager.cc:1905] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:24.028574 1566 ts_tablet_manager.cc:1905] T 8c623bd57a224ff784302af22e43806e P dcf1098d17e843dbb49cd767588538d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:24.030220 1564 ts_tablet_manager.cc:1918] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:24.030292 1564 log.cc:1196] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/8c623bd57a224ff784302af22e43806e
I20260808 00:41:24.030611 1564 ts_tablet_manager.cc:1939] T 8c623bd57a224ff784302af22e43806e P ff4eaeab2912444d9c01f22bac3bc313: Deleting consensus metadata
I20260808 00:41:24.031054 629 catalog_manager.cc:4841] TS ff4eaeab2912444d9c01f22bac3bc313 (127.1.208.3:33385): tablet 8c623bd57a224ff784302af22e43806e (table test-table [id=bbb2341dd9ff4c4281c85c911ee1c550]) successfully deleted
I20260808 00:41:24.037137 1563 tablet_replica.cc:330] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: stopping tablet replica
I20260808 00:41:24.037261 1563 raft_consensus.cc:2229] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:41:24.037389 1563 raft_consensus.cc:2258] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:24.037771 1563 ts_tablet_manager.cc:1905] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:24.039124 1563 ts_tablet_manager.cc:1918] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:24.039191 1563 log.cc:1196] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/8c623bd57a224ff784302af22e43806e
I20260808 00:41:24.039471 1563 ts_tablet_manager.cc:1939] T 8c623bd57a224ff784302af22e43806e P 5af1ca69dbe74a1188a7b48e8ce2e8bd: Deleting consensus metadata
I20260808 00:41:24.039822 629 catalog_manager.cc:4841] TS 5af1ca69dbe74a1188a7b48e8ce2e8bd (127.1.208.2:42575): tablet 8c623bd57a224ff784302af22e43806e (table test-table [id=bbb2341dd9ff4c4281c85c911ee1c550]) successfully deleted
W20260808 00:41:24.041903 633 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:33359 (error 108)
I20260808 00:41:24.042111 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1013
I20260808 00:41:24.042675 1513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.042260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:33359 (local address 127.0.0.1:41014)
0808 00:41:24.042609 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.042645 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:33359: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333}
W20260808 00:41:24.044209 633 catalog_manager.cc:4584] TS dcf1098d17e843dbb49cd767588538d3 (127.1.208.1:33359): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c623bd57a224ff784302af22e43806e: Network error: Client connection negotiation failed: client connection to 127.1.208.1:33359: connect: Connection refused (error 111)
I20260808 00:41:24.052066 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1271
I20260808 00:41:24.062146 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 603
2026-08-08T00:41:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2343 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T00:41:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:24Z Disabled control of system clock
I20260808 00:41:24.110008 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45891
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:45891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:24.191768 1572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:24.191984 1572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:24.192003 1572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:24.193091 1572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:24.193122 1572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:24.193133 1572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:24.193145 1572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:24.193158 1572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:24.194528 1572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:45891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:24.194744 1572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:24.194976 1572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:24.199513 1587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.199587 1572 server_base.cc:1034] running on GCE node
W20260808 00:41:24.199543 1585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.199548 1582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.200037 1572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:24.200302 1572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:24.201407 1572 hybrid_clock.cc:648] HybridClock initialized: now 1786149684201404 us; error 45 us; skew 500 ppm
I20260808 00:41:24.202581 1572 webserver.cc:460] Webserver started at http://127.1.208.62:43889/ using document root <none> and password file <none>
I20260808 00:41:24.202795 1572 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:24.202844 1572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:24.202976 1572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:24.204001 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "4646ebb54d25479c913f2d528be12786"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "953786ce71d2174e7592662741674885"
server_key_iv: "240c85708bdea0ef597190077c10fb25"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.204411 1572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "4646ebb54d25479c913f2d528be12786"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "953786ce71d2174e7592662741674885"
server_key_iv: "240c85708bdea0ef597190077c10fb25"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.205641 1572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:24.206310 1599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.206466 1572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:24.206549 1572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "4646ebb54d25479c913f2d528be12786"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "953786ce71d2174e7592662741674885"
server_key_iv: "240c85708bdea0ef597190077c10fb25"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.206616 1572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:24.215659 1572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:24.215912 1572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:24.216049 1572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:24.219846 1572 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:45891
I20260808 00:41:24.219877 1689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:45891 every 8 connection(s)
I20260808 00:41:24.220170 1572 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:24.221081 1690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:24.223311 1690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: Bootstrap starting.
I20260808 00:41:24.223832 1690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:24.224071 1690 log.cc:824] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:24.224478 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1572
I20260808 00:41:24.224599 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:24.224766 1690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: No bootstrap required, opened a new log
I20260808 00:41:24.224792 1856 external_mini_cluster.cc:1428] Setting key bf1dace45bf83d645fb84c0d6b4d62af
I20260808 00:41:24.227219 1690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } }
I20260808 00:41:24.227363 1690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:24.227406 1690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4646ebb54d25479c913f2d528be12786, State: Initialized, Role: FOLLOWER
I20260808 00:41:24.227528 1690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } }
I20260808 00:41:24.227605 1690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:24.227639 1690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:24.227681 1690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:24.228339 1690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } }
I20260808 00:41:24.228446 1690 leader_election.cc:302] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4646ebb54d25479c913f2d528be12786; no voters:
I20260808 00:41:24.228616 1690 leader_election.cc:288] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:24.228643 1706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:24.228843 1706 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [term 1 LEADER]: Becoming Leader. State: Replica: 4646ebb54d25479c913f2d528be12786, State: Running, Role: LEADER
I20260808 00:41:24.228883 1690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:24.229020 1706 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } }
I20260808 00:41:24.229456 1706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4646ebb54d25479c913f2d528be12786. Latest consensus state: current_term: 1 leader_uuid: "4646ebb54d25479c913f2d528be12786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } } }
I20260808 00:41:24.229465 1708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4646ebb54d25479c913f2d528be12786" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4646ebb54d25479c913f2d528be12786" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45891 } } }
I20260808 00:41:24.229636 1708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:24.229835 1706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:24.230203 1724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:24.230720 1724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:24.231571 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.225265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52238 (local address 127.1.208.62:45891)
0808 00:41:24.225483 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.225489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.225514 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:41:24.226016 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.226023 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.226087 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.226194 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:24.226428 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.227275 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.228371 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.229330 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.229770 (+ 440us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.230820 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:24.230862 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:24.230862 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:24.231064 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:24.231108 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:24.231124 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:24.231194 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.231289 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.231370 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":99,"threads_started":1}
I20260808 00:41:24.232263 1724 catalog_manager.cc:1347] Generated new cluster ID: e561da1182934af285ccedc0dda07833
I20260808 00:41:24.232318 1724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:24.246004 1724 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:24.246670 1724 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:24.268548 1724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: Generated new TSK 0
I20260808 00:41:24.268782 1724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:24.270888 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--builtin_ntp_servers=127.1.208.20:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:24.352612 1739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:24.352821 1739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:24.352842 1739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:24.354010 1739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:24.354053 1739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:24.354077 1739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:24.355417 1739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:24.355641 1739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:24.355943 1739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:24.361578 1749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.361586 1746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.361770 1739 server_base.cc:1034] running on GCE node
W20260808 00:41:24.361788 1754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.362035 1739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:24.362268 1739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:24.363406 1739 hybrid_clock.cc:648] HybridClock initialized: now 1786149684363392 us; error 27 us; skew 500 ppm
I20260808 00:41:24.364890 1739 webserver.cc:460] Webserver started at http://127.1.208.1:40889/ using document root <none> and password file <none>
I20260808 00:41:24.365157 1739 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:24.365226 1739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:24.365383 1739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:24.366782 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "5f88ac82db1d4e70bc1cd2085f4766cb"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "6194045481dd7ad8c4a011b8fd66afbc"
server_key_iv: "d3b3c94bd50fb36627f845bb7faaeba9"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.367156 1739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "5f88ac82db1d4e70bc1cd2085f4766cb"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "6194045481dd7ad8c4a011b8fd66afbc"
server_key_iv: "d3b3c94bd50fb36627f845bb7faaeba9"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.368613 1739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:41:24.369410 1761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.369565 1739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:24.369652 1739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "5f88ac82db1d4e70bc1cd2085f4766cb"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "6194045481dd7ad8c4a011b8fd66afbc"
server_key_iv: "d3b3c94bd50fb36627f845bb7faaeba9"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.369726 1739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:24.393510 1739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:24.393801 1739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:24.393905 1739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:24.394110 1739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:24.394398 1739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:24.394428 1739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.394456 1739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:24.394472 1739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.400031 1739 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:40453
I20260808 00:41:24.400094 1971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:40453 every 8 connection(s)
I20260808 00:41:24.400393 1739 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:24.405164 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.401036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:48227 (local address 127.1.208.62:45891)
0808 00:41:24.401120 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.401123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.401321 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:41:24.401854 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.401855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.401869 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.401920 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:24.402577 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.403065 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.404025 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.404228 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.404289 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.404967 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:24.404968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:24.404968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:24.404973 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:24.404985 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:24.404989 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:24.405025 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.405082 (+ 57us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.405103 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 00:41:24.405380 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.401015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45891 (local address 127.1.208.1:48227)
0808 00:41:24.401188 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.401224 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.401771 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.401977 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.401980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.401988 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:24.402496 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.402500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.403133 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.403134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.403919 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.403922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.404295 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.404296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.404406 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.404844 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:24.404844 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:24.404871 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:24.404894 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:24.405027 (+ 133us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:24.405029 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:24.405031 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.405075 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.405147 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260808 00:41:24.405848 1972 heartbeater.cc:347] Connected to a master server at 127.1.208.62:45891
I20260808 00:41:24.405932 1972 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:24.406103 1972 heartbeater.cc:511] Master 127.1.208.62:45891 requested a full tablet report, sending...
I20260808 00:41:24.406646 1628 ts_manager.cc:194] Registered new tserver with Master: 5f88ac82db1d4e70bc1cd2085f4766cb (127.1.208.1:40453)
I20260808 00:41:24.406911 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1739
I20260808 00:41:24.407008 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:41:24.407186 1856 external_mini_cluster.cc:1428] Setting key 4bbe2e7eabf750f2ee8a3b92d74c8596
I20260808 00:41:24.407433 1628 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.1:48227
I20260808 00:41:24.408278 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--builtin_ntp_servers=127.1.208.20:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:24.491343 1977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:24.491532 1977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:24.491552 1977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:24.492628 1977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:24.492662 1977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:24.492686 1977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:24.494026 1977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:24.494210 1977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:24.494467 1977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:24.499135 1985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.499155 1988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.499187 1986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.499500 1977 server_base.cc:1034] running on GCE node
I20260808 00:41:24.499692 1977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:24.499894 1977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:24.501026 1977 hybrid_clock.cc:648] HybridClock initialized: now 1786149684501014 us; error 28 us; skew 500 ppm
I20260808 00:41:24.502270 1977 webserver.cc:460] Webserver started at http://127.1.208.2:43345/ using document root <none> and password file <none>
I20260808 00:41:24.502496 1977 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:24.502540 1977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:24.502669 1977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:24.503681 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "791a9e7efd474432bf7e6cd31ebbc56b"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "3a3f6aee170a0225ee003ae326a68627"
server_key_iv: "49eb6a512063ba7873c9f4b9c4a4f50a"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.504031 1977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "791a9e7efd474432bf7e6cd31ebbc56b"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "3a3f6aee170a0225ee003ae326a68627"
server_key_iv: "49eb6a512063ba7873c9f4b9c4a4f50a"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.505254 1977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:24.505951 2000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.506094 1977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:24.506171 1977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "791a9e7efd474432bf7e6cd31ebbc56b"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "3a3f6aee170a0225ee003ae326a68627"
server_key_iv: "49eb6a512063ba7873c9f4b9c4a4f50a"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.506251 1977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:24.527093 1977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:24.527416 1977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:24.527545 1977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:24.527773 1977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:24.528108 1977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:24.528144 1977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.528165 1977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:24.528177 1977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.534243 1977 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:46227
I20260808 00:41:24.534296 2189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:46227 every 8 connection(s)
I20260808 00:41:24.534616 1977 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:41:24.539726 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.535347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:51363 (local address 127.1.208.62:45891)
0808 00:41:24.535404 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.535406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.535628 (+ 222us) server_negotiation.cc:407] Connection header received
0808 00:41:24.536201 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.536203 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.536215 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.536285 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:24.536952 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.537479 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.538450 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.538649 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.538722 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.539489 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:24.539490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:24.539490 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:24.539496 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:24.539511 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:24.539515 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:24.539569 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.539632 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.539658 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 00:41:24.539880 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.535312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45891 (local address 127.1.208.2:51363)
0808 00:41:24.535503 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.535545 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.536078 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.536373 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.536377 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.536386 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:24.536883 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.536888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.537557 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.537558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.538346 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.538348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.538722 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.538723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.538832 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.539341 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:24.539341 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:24.539372 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:24.539395 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:24.539573 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:24.539575 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:24.539577 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.539621 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.539699 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":75,"threads_started":1}
I20260808 00:41:24.540362 2192 heartbeater.cc:347] Connected to a master server at 127.1.208.62:45891
I20260808 00:41:24.540458 2192 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:24.540654 2192 heartbeater.cc:511] Master 127.1.208.62:45891 requested a full tablet report, sending...
I20260808 00:41:24.541047 1627 ts_manager.cc:194] Registered new tserver with Master: 791a9e7efd474432bf7e6cd31ebbc56b (127.1.208.2:46227)
I20260808 00:41:24.541476 1627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.2:51363
I20260808 00:41:24.543893 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 1977
I20260808 00:41:24.543977 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:24.544147 1856 external_mini_cluster.cc:1428] Setting key 101540c43d20280fc42a10c90c8cac0d
I20260808 00:41:24.544986 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--builtin_ntp_servers=127.1.208.20:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:24.628685 2197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:24.628923 2197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:24.628948 2197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:24.630177 2197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:24.630213 2197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:24.630237 2197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:24.631527 2197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:45891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:24.631713 2197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:24.631963 2197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:41:24.636879 2197 server_base.cc:1034] running on GCE node
W20260808 00:41:24.636862 2208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.636852 2205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:24.636862 2210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:24.637344 2197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:24.637600 2197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:24.638743 2197 hybrid_clock.cc:648] HybridClock initialized: now 1786149684638729 us; error 28 us; skew 500 ppm
I20260808 00:41:24.639948 2197 webserver.cc:460] Webserver started at http://127.1.208.3:36259/ using document root <none> and password file <none>
I20260808 00:41:24.640167 2197 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:24.640218 2197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:24.640359 2197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:24.641499 2197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "94f2fd4d2fb3eae0d8a384d8bbc557b8"
server_key_iv: "6f051614871af242b289e246b317625f"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.641841 2197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "94f2fd4d2fb3eae0d8a384d8bbc557b8"
server_key_iv: "6f051614871af242b289e246b317625f"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.643186 2197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:41:24.643934 2220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.644078 2197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:24.644161 2197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72"
format_stamp: "Formatted at 2026-08-08 00:41:24 on dist-test-slave-gjw7"
server_key: "94f2fd4d2fb3eae0d8a384d8bbc557b8"
server_key_iv: "6f051614871af242b289e246b317625f"
server_key_version: "encryptionkey@0"
I20260808 00:41:24.644227 2197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:24.652248 2197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:24.652491 2197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:24.652606 2197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:24.652791 2197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:24.653080 2197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:24.653115 2197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.653141 2197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:24.653164 2197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:24.658691 2197 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:46589
I20260808 00:41:24.658736 2403 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:46589 every 8 connection(s)
I20260808 00:41:24.659021 2197 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:41:24.660542 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2197
I20260808 00:41:24.660653 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:24.660857 1856 external_mini_cluster.cc:1428] Setting key bed8d7670599c0caf289aef291ef7d92
I20260808 00:41:24.664052 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.659707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:41301 (local address 127.1.208.62:45891)
0808 00:41:24.659757 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.659759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.660002 (+ 243us) server_negotiation.cc:407] Connection header received
0808 00:41:24.660499 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.660500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.660515 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.660586 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:24.661424 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.662037 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.662964 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.663149 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.663201 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.663847 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:24.663848 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:24.663849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:24.663853 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:24.663867 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:24.663871 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:24.663910 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.663971 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.663992 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:24.664229 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.659689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45891 (local address 127.1.208.3:41301)
0808 00:41:24.659886 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.659918 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.660389 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.660653 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.660657 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.660666 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:24.661251 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.661258 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.662114 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.662115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.662885 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.662888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.663207 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.663208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.663304 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.663731 (+ 427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:24.663732 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:24.663758 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:24.663780 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:24.663916 (+ 136us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:24.663918 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:24.663921 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.663966 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.664039 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260808 00:41:24.664623 2405 heartbeater.cc:347] Connected to a master server at 127.1.208.62:45891
I20260808 00:41:24.664708 2405 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:24.664863 2405 heartbeater.cc:511] Master 127.1.208.62:45891 requested a full tablet report, sending...
I20260808 00:41:24.665184 1627 ts_manager.cc:194] Registered new tserver with Master: 3a7ba36c0a0a45cda8d8a6569aa9db72 (127.1.208.3:46589)
I20260808 00:41:24.665577 1627 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.3:41301
I20260808 00:41:24.672497 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:24.676786 1698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.673905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52244 (local address 127.1.208.62:45891)
0808 00:41:24.673964 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.673966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.674087 (+ 121us) server_negotiation.cc:407] Connection header received
0808 00:41:24.674140 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.674140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.674151 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.674196 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:24.674338 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.674843 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.675587 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.675782 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.675835 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.676509 (+ 674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:24.676511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:24.676511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:24.676516 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:24.676531 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:24.676536 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:24.676620 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.676693 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.676703 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:41:24.678264 1627 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52244:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:24.679865 1627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:24.686050 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.682192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36228 (local address 127.1.208.2:46227)
0808 00:41:24.682413 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.682418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.682524 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:41:24.682580 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.682582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.682598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.682691 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:24.682875 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.683941 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685126 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685664 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685727 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685820 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.685924 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.685963 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":106,"threads_started":1}
I20260808 00:41:24.686038 2430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.681926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:46227 (local address 127.0.0.1:36228)
0808 00:41:24.682447 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.682455 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.682507 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.682766 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.682767 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.682769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:24.682804 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.682805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.684021 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.684022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.685058 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.685060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685744 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.685820 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685843 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.685911 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.685962 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260808 00:41:24.686215 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.682015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.1.208.3:46589)
0808 00:41:24.682383 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.682386 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.682396 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:24.682440 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.682441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.682453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.682525 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:24.682769 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.683897 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.684950 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685686 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685758 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685842 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.686093 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.686131 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":101,"threads_started":1}
I20260808 00:41:24.686228 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.682193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39780 (local address 127.1.208.1:40453)
0808 00:41:24.682355 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.682361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.682575 (+ 214us) server_negotiation.cc:407] Connection header received
0808 00:41:24.682619 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.682621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.682635 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.682703 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:24.682907 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.683601 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.684924 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685685 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685756 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685842 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.686093 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.686137 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":79,"threads_started":1}
I20260808 00:41:24.686249 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.681924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:46589 (local address 127.0.0.1:47772)
0808 00:41:24.682212 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.682254 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.682337 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.682606 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.682609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.682612 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:24.682678 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.682680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.683999 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.684001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.684866 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.684869 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685880 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.685952 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685958 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.686005 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.686049 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":41,"thread_start_us":80,"threads_started":1}
I20260808 00:41:24.686409 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.681926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40453 (local address 127.0.0.1:39780)
0808 00:41:24.682464 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.682474 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.682521 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.682774 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.682775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.682777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:24.682821 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.682833 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.683677 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.683678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.684854 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.684857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.685879 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.685880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.685952 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.685957 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.686005 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.686048 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"mutex_wait_us":8,"thread_start_us":105,"threads_started":1}
I20260808 00:41:24.686944 2306 tablet_service.cc:1467] Processing CreateTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf (DEFAULT_TABLE table=test-table [id=ba28013544f248ea9464d5f7669eb388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:24.687178 2084 tablet_service.cc:1467] Processing CreateTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf (DEFAULT_TABLE table=test-table [id=ba28013544f248ea9464d5f7669eb388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:24.687179 1848 tablet_service.cc:1467] Processing CreateTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf (DEFAULT_TABLE table=test-table [id=ba28013544f248ea9464d5f7669eb388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:24.687340 2306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b95afb1ad934fe4b2f0d786341562bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:24.687516 2084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b95afb1ad934fe4b2f0d786341562bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:24.687516 1848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b95afb1ad934fe4b2f0d786341562bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:24.690793 2437 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Bootstrap starting.
I20260808 00:41:24.690922 2434 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Bootstrap starting.
I20260808 00:41:24.691382 2437 tablet_bootstrap.cc:654] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:24.691473 2434 tablet_bootstrap.cc:654] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:24.691666 2437 log.cc:824] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:24.691728 2434 log.cc:824] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:24.692008 2438 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Bootstrap starting.
I20260808 00:41:24.692396 2434 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: No bootstrap required, opened a new log
I20260808 00:41:24.692435 2437 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: No bootstrap required, opened a new log
I20260808 00:41:24.692451 2434 ts_tablet_manager.cc:1397] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:24.692492 2437 ts_tablet_manager.cc:1397] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:24.694401 2434 raft_consensus.cc:348] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.694430 2437 raft_consensus.cc:348] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.694533 2434 raft_consensus.cc:374] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:24.694554 2434 raft_consensus.cc:729] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a7ba36c0a0a45cda8d8a6569aa9db72, State: Initialized, Role: FOLLOWER
I20260808 00:41:24.694550 2437 raft_consensus.cc:374] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:24.694582 2437 raft_consensus.cc:729] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f88ac82db1d4e70bc1cd2085f4766cb, State: Initialized, Role: FOLLOWER
I20260808 00:41:24.694675 2434 consensus_queue.cc:260] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.694695 2437 consensus_queue.cc:260] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.694958 2437 ts_tablet_manager.cc:1428] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:41:24.694970 2405 heartbeater.cc:503] Master 127.1.208.62:45891 was elected leader, sending a full tablet report...
I20260808 00:41:24.694998 1972 heartbeater.cc:503] Master 127.1.208.62:45891 was elected leader, sending a full tablet report...
I20260808 00:41:24.694931 2434 ts_tablet_manager.cc:1428] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:41:24.695570 2438 tablet_bootstrap.cc:654] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:24.696084 2438 log.cc:824] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:24.696736 2438 tablet_bootstrap.cc:492] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: No bootstrap required, opened a new log
I20260808 00:41:24.696791 2438 ts_tablet_manager.cc:1397] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260808 00:41:24.698679 2438 raft_consensus.cc:348] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.698814 2438 raft_consensus.cc:374] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:24.698836 2438 raft_consensus.cc:729] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791a9e7efd474432bf7e6cd31ebbc56b, State: Initialized, Role: FOLLOWER
I20260808 00:41:24.698915 2438 consensus_queue.cc:260] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.699092 2438 ts_tablet_manager.cc:1428] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:41:24.699165 2192 heartbeater.cc:503] Master 127.1.208.62:45891 was elected leader, sending a full tablet report...
W20260808 00:41:24.785173 2193 tablet.cc:2369] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:24.854758 2446 raft_consensus.cc:482] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:24.854897 2446 raft_consensus.cc:504] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.855230 2446 leader_election.cc:288] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a7ba36c0a0a45cda8d8a6569aa9db72 (127.1.208.3:46589), 5f88ac82db1d4e70bc1cd2085f4766cb (127.1.208.1:40453)
I20260808 00:41:24.858295 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.855643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:37415 (local address 127.1.208.1:40453)
0808 00:41:24.855692 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.855694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.855807 (+ 113us) server_negotiation.cc:407] Connection header received
0808 00:41:24.855868 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.855868 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.855880 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.855953 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:24.856152 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.856618 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.857615 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.857972 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.858032 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.858043 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.858188 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.858221 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:41:24.858302 2431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.855647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:51455 (local address 127.1.208.3:46589)
0808 00:41:24.855701 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.855703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.855715 (+ 12us) server_negotiation.cc:407] Connection header received
0808 00:41:24.855757 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.855758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.855769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.855833 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:24.856044 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.856598 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.857624 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.857975 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.858037 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.858047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.858186 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.858229 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:41:24.858325 2195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.855504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:46589 (local address 127.1.208.2:51455)
0808 00:41:24.855565 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.855579 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.855649 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.855855 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.855856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.855858 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:24.855947 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.855949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.856681 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.856682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.857543 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.857545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.858050 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.858051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.858124 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.858127 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.858173 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.858215 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":1}
I20260808 00:41:24.858342 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.855504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40453 (local address 127.1.208.2:37415)
0808 00:41:24.855732 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.855742 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:24.855811 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:24.855984 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:24.855985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:24.855988 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:24.856073 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.856075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.856697 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.856698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.857532 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:24.857534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.858071 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.858072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:24.858126 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.858128 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:24.858174 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 00:41:24.858217 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":18,"thread_start_us":128,"threads_started":1}
I20260808 00:41:24.858686 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b95afb1ad934fe4b2f0d786341562bf" candidate_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" is_pre_election: true
I20260808 00:41:24.858716 2333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b95afb1ad934fe4b2f0d786341562bf" candidate_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" is_pre_election: true
I20260808 00:41:24.858848 2333 raft_consensus.cc:2454] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791a9e7efd474432bf7e6cd31ebbc56b in term 0.
I20260808 00:41:24.858852 1891 raft_consensus.cc:2454] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791a9e7efd474432bf7e6cd31ebbc56b in term 0.
I20260808 00:41:24.859083 2008 leader_election.cc:302] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f88ac82db1d4e70bc1cd2085f4766cb, 791a9e7efd474432bf7e6cd31ebbc56b; no voters:
I20260808 00:41:24.859206 2446 raft_consensus.cc:2781] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:24.859263 2446 raft_consensus.cc:482] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:24.859287 2446 raft_consensus.cc:3037] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:24.860011 2446 raft_consensus.cc:504] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.860140 2446 leader_election.cc:288] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [CANDIDATE]: Term 1 election: Requested vote from peers 3a7ba36c0a0a45cda8d8a6569aa9db72 (127.1.208.3:46589), 5f88ac82db1d4e70bc1cd2085f4766cb (127.1.208.1:40453)
I20260808 00:41:24.860303 2333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b95afb1ad934fe4b2f0d786341562bf" candidate_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72"
I20260808 00:41:24.860383 2333 raft_consensus.cc:3037] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:24.861104 2333 raft_consensus.cc:2454] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791a9e7efd474432bf7e6cd31ebbc56b in term 1.
I20260808 00:41:24.861229 1891 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b95afb1ad934fe4b2f0d786341562bf" candidate_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb"
I20260808 00:41:24.861347 2006 leader_election.cc:302] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a7ba36c0a0a45cda8d8a6569aa9db72, 791a9e7efd474432bf7e6cd31ebbc56b; no voters:
I20260808 00:41:24.861335 1891 raft_consensus.cc:3037] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:24.861479 2446 raft_consensus.cc:2781] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:24.861627 2446 raft_consensus.cc:686] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 1 LEADER]: Becoming Leader. State: Replica: 791a9e7efd474432bf7e6cd31ebbc56b, State: Running, Role: LEADER
I20260808 00:41:24.861725 2446 consensus_queue.cc:237] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } }
I20260808 00:41:24.862178 1891 raft_consensus.cc:2454] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791a9e7efd474432bf7e6cd31ebbc56b in term 1.
I20260808 00:41:24.862672 1627 catalog_manager.cc:5495] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b reported cstate change: term changed from 0 to 1, leader changed from <none> to 791a9e7efd474432bf7e6cd31ebbc56b (127.1.208.2). New cstate: current_term: 1 leader_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791a9e7efd474432bf7e6cd31ebbc56b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 46227 } health_report { overall_health: HEALTHY } } }
W20260808 00:41:24.901007 1975 tablet.cc:2369] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 00:41:24.909578 2407 tablet.cc:2369] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:24.910772 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.908389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36240 (local address 127.1.208.2:46227)
0808 00:41:24.908436 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:24.908438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:24.908447 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:24.908502 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:24.908503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:24.908524 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:24.908582 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:24.908714 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.909212 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.910157 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:24.910357 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:24.910410 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:24.910509 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:24.910595 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:24.910657 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:24.910691 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 00:41:24.910699 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260808 00:41:24.912546 2333 raft_consensus.cc:1261] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 1 FOLLOWER]: Refusing update from remote peer 791a9e7efd474432bf7e6cd31ebbc56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:24.912572 1891 raft_consensus.cc:1261] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb [term 1 FOLLOWER]: Refusing update from remote peer 791a9e7efd474432bf7e6cd31ebbc56b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:24.912894 2446 consensus_queue.cc:1035] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a7ba36c0a0a45cda8d8a6569aa9db72" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 46589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:24.913036 2450 consensus_queue.cc:1035] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f88ac82db1d4e70bc1cd2085f4766cb" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 40453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:24.915129 2461 mvcc.cc:205] Tried to move back new op lower bound from 7316069109399093248 to 7316069109194170368. Current Snapshot: MvccSnapshot[applied={T|T < 7316069109399093248}]
I20260808 00:41:24.917946 1627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:24.918028 1627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52244:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:24.918692 1627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4646ebb54d25479c913f2d528be12786: Sending DeleteTablet for 3 replicas of tablet 5b95afb1ad934fe4b2f0d786341562bf
I20260808 00:41:24.919008 2306 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='slave'} at 127.0.0.1:47772
I20260808 00:41:24.919065 2084 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='slave'} at 127.0.0.1:36228
I20260808 00:41:24.919238 2474 tablet_replica.cc:330] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: stopping tablet replica
I20260808 00:41:24.919224 1848 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b95afb1ad934fe4b2f0d786341562bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:24 UTC) from {username='slave'} at 127.0.0.1:39780
I20260808 00:41:24.919324 2474 raft_consensus.cc:2229] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:24.919431 2474 raft_consensus.cc:2258] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:24.919631 2477 tablet_replica.cc:330] T 5b95afb1ad934fe4b2f0d786341562bf P 5f88ac82db1d4e70bc1cd2085f4766cb: stopping tablet replica
I20260808 00:41:24.919783 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1739
I20260808 00:41:24.941056 2474 ts_tablet_manager.cc:1905] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:24.942780 2474 ts_tablet_manager.cc:1918] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:24.942890 2474 log.cc:1196] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/5b95afb1ad934fe4b2f0d786341562bf
I20260808 00:41:24.943159 2474 ts_tablet_manager.cc:1939] T 5b95afb1ad934fe4b2f0d786341562bf P 3a7ba36c0a0a45cda8d8a6569aa9db72: Deleting consensus metadata
I20260808 00:41:24.943161 2475 tablet_replica.cc:330] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: stopping tablet replica
I20260808 00:41:24.943419 2475 raft_consensus.cc:2229] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 1 LEADER]: Raft consensus shutting down.
I20260808 00:41:24.943641 2475 raft_consensus.cc:2258] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:24.943657 1602 catalog_manager.cc:4841] TS 3a7ba36c0a0a45cda8d8a6569aa9db72 (127.1.208.3:46589): tablet 5b95afb1ad934fe4b2f0d786341562bf (table test-table [id=ba28013544f248ea9464d5f7669eb388]) successfully deleted
I20260808 00:41:24.944152 2475 ts_tablet_manager.cc:1905] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:24.945423 2475 ts_tablet_manager.cc:1918] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:24.945493 2475 log.cc:1196] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/5b95afb1ad934fe4b2f0d786341562bf
I20260808 00:41:24.945739 2475 ts_tablet_manager.cc:1939] T 5b95afb1ad934fe4b2f0d786341562bf P 791a9e7efd474432bf7e6cd31ebbc56b: Deleting consensus metadata
I20260808 00:41:24.946141 1600 catalog_manager.cc:4841] TS 791a9e7efd474432bf7e6cd31ebbc56b (127.1.208.2:46227): tablet 5b95afb1ad934fe4b2f0d786341562bf (table test-table [id=ba28013544f248ea9464d5f7669eb388]) successfully deleted
W20260808 00:41:24.946478 1603 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:40453 (error 108)
I20260808 00:41:24.947007 2422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:24.946866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:40453 (local address 127.0.0.1:39784)
0808 00:41:24.946914 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:24.946972 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:41:24.947091 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1977
W20260808 00:41:24.947139 1603 catalog_manager.cc:4584] TS 5f88ac82db1d4e70bc1cd2085f4766cb (127.1.208.1:40453): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b95afb1ad934fe4b2f0d786341562bf: Network error: Client connection negotiation failed: client connection to 127.1.208.1:40453: connect: Connection refused (error 111)
I20260808 00:41:24.968333 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2197
I20260808 00:41:24.979817 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 1572
2026-08-08T00:41:24Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (912 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:25 dist-test-slave-gjw7 krb5kdc[2485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:25 dist-test-slave-gjw7 krb5kdc[2485](info): set up 2 sockets
Aug 08 00:41:25 dist-test-slave-gjw7 krb5kdc[2485](info): commencing operation
krb5kdc: starting...
W20260808 00:41:26.183280 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:26.299757 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:33333
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:33333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:26.380779 2510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:26.380981 2510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:26.381011 2510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:26.382143 2510 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:26.382184 2510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:26.382203 2510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:26.382218 2510 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:26.382233 2510 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:26.383687 2510 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:33333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:33333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:26.383931 2510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:26.384141 2510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:26.388684 2521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.388686 2520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.388720 2523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:26.388706 2510 server_base.cc:1034] running on GCE node
I20260808 00:41:26.389171 2510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:26.389467 2510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:26.390609 2510 hybrid_clock.cc:648] HybridClock initialized: now 1786149686390597 us; error 30 us; skew 500 ppm
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:26.392952 2510 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:26.393389 2510 webserver.cc:460] Webserver started at http://127.1.208.62:42607/ using document root <none> and password file <none>
I20260808 00:41:26.393602 2510 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:26.393649 2510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:26.393770 2510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:26.394773 2510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "66cdf098c3a3a369a8c6d695133763fc"
server_key_iv: "a63627369f1efe3471be595dd0febc54"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.395097 2510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "66cdf098c3a3a369a8c6d695133763fc"
server_key_iv: "a63627369f1efe3471be595dd0febc54"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.396391 2510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:26.397035 2536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.397185 2510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:26.397267 2510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "66cdf098c3a3a369a8c6d695133763fc"
server_key_iv: "a63627369f1efe3471be595dd0febc54"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.397380 2510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:26.427335 2510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:26.429281 2510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:26.429445 2510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:26.433259 2510 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:33333
I20260808 00:41:26.433346 2628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:33333 every 8 connection(s)
I20260808 00:41:26.433655 2510 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:26.434480 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2510
I20260808 00:41:26.434506 2631 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:26.434588 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:26.434765 1856 external_mini_cluster.cc:1428] Setting key 4ce7dab2e989894382ecfcbf391d49d6
I20260808 00:41:26.437565 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: Bootstrap starting.
I20260808 00:41:26.438167 2631 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:26.438436 2631 log.cc:824] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:26.439163 2631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: No bootstrap required, opened a new log
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:26.441223 2631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } }
I20260808 00:41:26.441398 2631 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:26.441425 2631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c603c0f9cfe4c33bd6e4753fe1cfd3a, State: Initialized, Role: FOLLOWER
I20260808 00:41:26.441547 2631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } }
I20260808 00:41:26.441629 2631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:26.441660 2631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:26.441713 2631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:26.442381 2631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } }
I20260808 00:41:26.442498 2631 leader_election.cc:302] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c603c0f9cfe4c33bd6e4753fe1cfd3a; no voters:
I20260808 00:41:26.442664 2631 leader_election.cc:288] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:26.442718 2642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:26.442873 2642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [term 1 LEADER]: Becoming Leader. State: Replica: 6c603c0f9cfe4c33bd6e4753fe1cfd3a, State: Running, Role: LEADER
I20260808 00:41:26.442979 2631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:26.443071 2642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } }
I20260808 00:41:26.443445 2642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c603c0f9cfe4c33bd6e4753fe1cfd3a. Latest consensus state: current_term: 1 leader_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } } }
I20260808 00:41:26.443503 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c603c0f9cfe4c33bd6e4753fe1cfd3a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 33333 } } }
I20260808 00:41:26.443588 2642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:26.443594 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:26.444356 2657 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:26.444886 2657 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:26.445657 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.435314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48166 (local address 127.1.208.62:33333)
0808 00:41:26.435493 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.435501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.435527 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:41:26.435995 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.436002 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.436049 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.436188 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:26.436802 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.437780 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.438571 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.439099 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.439226 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.441719 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:26.441723 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:26.441723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:26.441735 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:26.443592 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.444035 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.444038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.444039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.444162 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.444401 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.444402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.444402 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.444769 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:26.444917 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.445275 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.445460 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260808 00:41:26.449702 2657 catalog_manager.cc:1347] Generated new cluster ID: b7ad156f9b144c58ad4cc1d3222d18af
I20260808 00:41:26.449755 2657 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:26.463446 2657 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:26.464221 2657 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:26.467703 2657 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: Generated new TSK 0
I20260808 00:41:26.467885 2657 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.208.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
Entry for principal kudu/127.1.208.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab.
I20260808 00:41:26.497452 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:26.582185 2681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:26.582448 2681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:26.582479 2681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:26.583812 2681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:26.583861 2681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:26.583886 2681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:26.585312 2681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.1.keytab
--principal=kudu/127.1.208.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:26.585525 2681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:26.585739 2681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:26.590436 2694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.590436 2691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.590473 2692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:26.590754 2681 server_base.cc:1034] running on GCE node
I20260808 00:41:26.590961 2681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:26.591173 2681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:26.592325 2681 hybrid_clock.cc:648] HybridClock initialized: now 1786149686592308 us; error 28 us; skew 500 ppm
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:26.594750 2681 init.cc:377] Logged in from keytab as kudu/127.1.208.1@KRBTEST.COM (short username kudu)
I20260808 00:41:26.595148 2681 webserver.cc:460] Webserver started at http://127.1.208.1:42653/ using document root <none> and password file <none>
I20260808 00:41:26.595386 2681 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:26.595439 2681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:26.595561 2681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:26.596582 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "91ec10a811394c7baea22781b9cbb10c"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "ba063f9897ff8c00b755024b2a122432"
server_key_iv: "bbf922e6af7ae5068f8b2b57479bfc86"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.596910 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "91ec10a811394c7baea22781b9cbb10c"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "ba063f9897ff8c00b755024b2a122432"
server_key_iv: "bbf922e6af7ae5068f8b2b57479bfc86"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.598181 2681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:26.598850 2704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.599038 2681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:26.599123 2681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "91ec10a811394c7baea22781b9cbb10c"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "ba063f9897ff8c00b755024b2a122432"
server_key_iv: "bbf922e6af7ae5068f8b2b57479bfc86"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.599182 2681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:26.612192 2681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:26.614065 2681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:26.614177 2681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:26.614362 2681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:26.614607 2681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:26.614637 2681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.614658 2681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:26.614673 2681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.620105 2681 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:35015
I20260808 00:41:26.620178 2909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:35015 every 8 connection(s)
I20260808 00:41:26.620467 2681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:26.623023 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2681
I20260808 00:41:26.623121 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:41:26.623297 1856 external_mini_cluster.cc:1428] Setting key 902c15b2bdd5a62a9d7f286100380e18
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.1@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:26.629818 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.621136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:59481 (local address 127.1.208.62:33333)
0808 00:41:26.621190 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.621192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.621408 (+ 216us) server_negotiation.cc:407] Connection header received
0808 00:41:26.621920 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.621921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.621933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.621987 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:26.622903 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.623535 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.624567 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.624785 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.624841 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.627091 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:26.627093 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:26.627093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:26.627101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:26.628458 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.628884 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.628886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.628886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.628950 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.629152 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.629153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.629154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.629284 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:26.629376 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.629639 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.629728 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:41:26.629979 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.621104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33333 (local address 127.1.208.1:59481)
0808 00:41:26.621251 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.621321 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.621816 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.622057 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.622061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.622601 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:26.622784 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.622789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.623610 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.623612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.624480 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.624483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.624862 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.624863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.624975 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.625454 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:26.625463 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:26.626991 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:26.628574 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.628583 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.628585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.628812 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.629012 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.629012 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.629012 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.629102 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.629383 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:26.629391 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:26.629522 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.629609 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.629760 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260808 00:41:26.630409 2911 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33333
I20260808 00:41:26.630496 2911 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:26.630659 2911 heartbeater.cc:511] Master 127.1.208.62:33333 requested a full tablet report, sending...
I20260808 00:41:26.631179 2563 ts_manager.cc:194] Registered new tserver with Master: 91ec10a811394c7baea22781b9cbb10c (127.1.208.1:35015)
I20260808 00:41:26.631784 2563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.1@KRBTEST.COM'} at 127.1.208.1:59481
WARNING: no policy specified for kudu/127.1.208.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
Entry for principal kudu/127.1.208.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab.
I20260808 00:41:26.649533 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:26.731381 2924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:26.731570 2924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:26.731591 2924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:26.732649 2924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:26.732681 2924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:26.732704 2924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:26.734163 2924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.2.keytab
--principal=kudu/127.1.208.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:26.734359 2924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:26.734537 2924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:26.739238 2939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.739339 2937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.739270 2936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:26.739485 2924 server_base.cc:1034] running on GCE node
I20260808 00:41:26.739665 2924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:26.739861 2924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:26.740984 2924 hybrid_clock.cc:648] HybridClock initialized: now 1786149686740976 us; error 29 us; skew 500 ppm
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:26.743438 2924 init.cc:377] Logged in from keytab as kudu/127.1.208.2@KRBTEST.COM (short username kudu)
I20260808 00:41:26.743845 2924 webserver.cc:460] Webserver started at http://127.1.208.2:40607/ using document root <none> and password file <none>
I20260808 00:41:26.744088 2924 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:26.744134 2924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:26.744252 2924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:26.745252 2924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "77397a8fbf9f4c3a83e2c15c05832a77"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "e98049467b3396d1c1ca52255af2bdd0"
server_key_iv: "072665f6f241ac6318f77aa41bb71b7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.745640 2924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "77397a8fbf9f4c3a83e2c15c05832a77"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "e98049467b3396d1c1ca52255af2bdd0"
server_key_iv: "072665f6f241ac6318f77aa41bb71b7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.746869 2924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:26.747526 2952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.747669 2924 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:26.747756 2924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "77397a8fbf9f4c3a83e2c15c05832a77"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "e98049467b3396d1c1ca52255af2bdd0"
server_key_iv: "072665f6f241ac6318f77aa41bb71b7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.747819 2924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:26.757997 2924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:26.759625 2924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:26.759747 2924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:26.759934 2924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:26.760210 2924 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:26.760243 2924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.760272 2924 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:26.760295 2924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.765950 2924 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:41481
I20260808 00:41:26.766016 3131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:41481 every 8 connection(s)
I20260808 00:41:26.766287 2924 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.2@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:26.774470 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.766966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:53679 (local address 127.1.208.62:33333)
0808 00:41:26.767016 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.767019 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.767209 (+ 190us) server_negotiation.cc:407] Connection header received
0808 00:41:26.767681 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.767682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.767692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.767738 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:26.768616 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.769101 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.770101 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.770283 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.770341 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.772202 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:26.772204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:26.772204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:26.772210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:26.773477 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.773800 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.773801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.773801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.773848 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.774015 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.774015 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.774016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.774106 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:26.774149 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.774346 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.774405 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:41:26.774605 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 2924
I20260808 00:41:26.774721 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:26.774698 3134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.766943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33333 (local address 127.1.208.2:53679)
0808 00:41:26.767100 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.767135 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.767603 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.767804 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.767807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.768357 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:26.768549 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.768554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.769167 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.769167 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.770033 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.770037 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.770343 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.770344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.770442 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.770861 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:26.770869 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:26.772123 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:26.773536 (+ 1413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.773542 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.773543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.773748 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.773889 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.773890 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.773890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.773976 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.774145 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:26.774147 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:26.774250 (+ 103us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.774321 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.774475 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260808 00:41:26.774909 1856 external_mini_cluster.cc:1428] Setting key c3aa636c5119bcfbebe0780f70d897fa
I20260808 00:41:26.775271 3132 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33333
I20260808 00:41:26.775370 3132 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:26.775604 3132 heartbeater.cc:511] Master 127.1.208.62:33333 requested a full tablet report, sending...
I20260808 00:41:26.776036 2563 ts_manager.cc:194] Registered new tserver with Master: 77397a8fbf9f4c3a83e2c15c05832a77 (127.1.208.2:41481)
I20260808 00:41:26.776399 2563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.2@KRBTEST.COM'} at 127.1.208.2:53679
WARNING: no policy specified for kudu/127.1.208.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
Entry for principal kudu/127.1.208.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab.
I20260808 00:41:26.799197 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:46459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:26.880923 3141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:26.881131 3141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:26.881155 3141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:26.882287 3141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:26.882329 3141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:26.882354 3141 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:26.883798 3141 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.3.keytab
--principal=kudu/127.1.208.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:33333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:26.884030 3141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:26.884239 3141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:26.889037 3152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.889046 3148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:26.889034 3149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:26.889220 3141 server_base.cc:1034] running on GCE node
I20260808 00:41:26.889492 3141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:26.889735 3141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:26.890903 3141 hybrid_clock.cc:648] HybridClock initialized: now 1786149686890878 us; error 34 us; skew 500 ppm
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:26.893416 3141 init.cc:377] Logged in from keytab as kudu/127.1.208.3@KRBTEST.COM (short username kudu)
I20260808 00:41:26.893864 3141 webserver.cc:460] Webserver started at http://127.1.208.3:39517/ using document root <none> and password file <none>
I20260808 00:41:26.894073 3141 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:26.894125 3141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:26.894263 3141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:26.895273 3141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "c9cd9774d89d493f89d2b51b86ef68d9"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "002b2945feb5b4d1711135b23dd7b51d"
server_key_iv: "c19732df8b12c9c00c20e5e4ae48be27"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.895630 3141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "c9cd9774d89d493f89d2b51b86ef68d9"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "002b2945feb5b4d1711135b23dd7b51d"
server_key_iv: "c19732df8b12c9c00c20e5e4ae48be27"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.896957 3141 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:41:26.897746 3166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.897881 3141 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.897977 3141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "c9cd9774d89d493f89d2b51b86ef68d9"
format_stamp: "Formatted at 2026-08-08 00:41:26 on dist-test-slave-gjw7"
server_key: "002b2945feb5b4d1711135b23dd7b51d"
server_key_iv: "c19732df8b12c9c00c20e5e4ae48be27"
server_key_version: "encryptionkey@0"
I20260808 00:41:26.898041 3141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:26.908653 3141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:26.910375 3141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:26.910507 3141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:26.910707 3141 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:26.911010 3141 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:26.911051 3141 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.911082 3141 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:26.911104 3141 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:26.916877 3141 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:33623
I20260808 00:41:26.916913 3372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:33623 every 8 connection(s)
I20260808 00:41:26.917209 3141 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
Aug 08 00:41:26 dist-test-slave-gjw7 krb5kdc[2485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149686, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.3@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:26.924486 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3141
I20260808 00:41:26.924592 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:26.924796 1856 external_mini_cluster.cc:1428] Setting key 2a01036fd49f9efb5b3b1f9817fd9f37
I20260808 00:41:26.925892 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.917940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:46545 (local address 127.1.208.62:33333)
0808 00:41:26.917982 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.917984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.918192 (+ 208us) server_negotiation.cc:407] Connection header received
0808 00:41:26.918655 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.918656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.918665 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.918716 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:26.919584 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.920077 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.920965 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.921135 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.921178 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.923072 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:26.923074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:26.923074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:26.923081 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:26.924301 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.924820 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.924821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.924821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.924894 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.925128 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.925129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.925129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.925271 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:26.925371 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.925709 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.925803 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:26.926038 3375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.917907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:33333 (local address 127.1.208.3:46545)
0808 00:41:26.918072 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.918113 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.918582 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.918777 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.918780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.919328 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:26.919514 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.919519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.920135 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.920136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.920904 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.920906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.921186 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.921187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.921280 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.921690 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:26.921698 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:26.922984 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:26.924420 (+ 1436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.924429 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.924432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.924745 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.924944 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:26.924945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:26.924945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:26.925065 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:26.925413 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:26.925420 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:26.925573 (+ 153us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.925665 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.925824 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260808 00:41:26.926481 3373 heartbeater.cc:347] Connected to a master server at 127.1.208.62:33333
I20260808 00:41:26.926558 3373 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:26.926700 3373 heartbeater.cc:511] Master 127.1.208.62:33333 requested a full tablet report, sending...
I20260808 00:41:26.927018 2563 ts_manager.cc:194] Registered new tserver with Master: c9cd9774d89d493f89d2b51b86ef68d9 (127.1.208.3:33623)
I20260808 00:41:26.927408 2563 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.208.3@KRBTEST.COM'} at 127.1.208.3:46545
I20260808 00:41:26.936558 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:26.943470 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.937988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48168 (local address 127.1.208.62:33333)
0808 00:41:26.938049 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.938051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.938141 (+ 90us) server_negotiation.cc:407] Connection header received
0808 00:41:26.938202 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.938202 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.938213 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.938268 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:26.938820 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.939297 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.940015 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.940200 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.940260 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.941324 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:26.941325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:26.941325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:26.941332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:26.942408 (+ 1076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.942715 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.942716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.942717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.942773 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:26.942955 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:26.942956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:26.942956 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:26.943105 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:26.943211 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.943304 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.943378 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 00:41:26.945016 2563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:26.946687 2563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:26.952978 3392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41481 (local address 127.0.0.1:32824)
0808 00:41:26.948996 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.949030 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.949091 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.949466 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.949469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.949474 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:26.949560 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.949561 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.950886 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.950887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.952026 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.952028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.952677 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.952769 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.952784 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.952853 (+ 69us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.952904 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":25,"thread_start_us":85,"threads_started":1}
I20260808 00:41:26.953027 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32824 (local address 127.1.208.2:41481)
0808 00:41:26.948995 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.949000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.949162 (+ 162us) server_negotiation.cc:407] Connection header received
0808 00:41:26.949213 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.949260 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.949276 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.949368 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:26.949642 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.950791 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952098 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.952594 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952662 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.952722 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.952891 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.952940 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":100,"threads_started":1}
I20260808 00:41:26.953890 3390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:33623 (local address 127.0.0.1:50336)
0808 00:41:26.948983 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.949024 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.949087 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.949466 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.949469 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.949471 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:26.949559 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.949561 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.952122 (+ 2561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.952812 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.952814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.953660 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.953661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.953730 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.953735 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.953785 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.953826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":7,"thread_start_us":49,"threads_started":1}
I20260808 00:41:26.954013 3024 tablet_service.cc:1467] Processing CreateTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba (DEFAULT_TABLE table=test-table [id=039ed847a0b44cecbd2ed1d63d752b30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:26.954145 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.1.208.3:33623)
0808 00:41:26.949197 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.949202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.949211 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:26.949246 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.949248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.949264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.949385 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:26.949643 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.950783 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952902 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.953585 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.953658 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.953714 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.953994 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.954062 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":105,"threads_started":1}
I20260808 00:41:26.954378 3024 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481f5cf306a4468688fef9fc1fa0e3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:26.954752 3247 tablet_service.cc:1467] Processing CreateTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba (DEFAULT_TABLE table=test-table [id=039ed847a0b44cecbd2ed1d63d752b30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:26.955032 3247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481f5cf306a4468688fef9fc1fa0e3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:26.957077 3398 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Bootstrap starting.
I20260808 00:41:26.957369 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35015 (local address 127.0.0.1:49420)
0808 00:41:26.949355 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:26.949363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:26.949411 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:26.949679 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:26.949680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:26.949682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:26.949735 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.949736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.950672 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.950673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.952024 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:26.952028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.957105 (+ 5077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.957106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:26.957180 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.957185 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:26.957238 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 00:41:26.957279 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"mutex_wait_us":4,"thread_start_us":40,"threads_started":1}
I20260808 00:41:26.957679 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:26.948974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49420 (local address 127.1.208.1:35015)
0808 00:41:26.949138 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:26.949141 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:26.949464 (+ 323us) server_negotiation.cc:407] Connection header received
0808 00:41:26.949512 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:26.949514 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:26.949529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:26.949597 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:26.949802 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.950584 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.952761 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:26.953497 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:26.953569 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:26.953794 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:26.957550 (+ 3756us) server_negotiation.cc:299] Negotiation successful
0808 00:41:26.957599 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":84,"threads_started":1}
I20260808 00:41:26.957878 3401 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Bootstrap starting.
I20260808 00:41:26.957959 3398 tablet_bootstrap.cc:654] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:26.958268 2797 tablet_service.cc:1467] Processing CreateTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba (DEFAULT_TABLE table=test-table [id=039ed847a0b44cecbd2ed1d63d752b30]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:26.958554 2797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 481f5cf306a4468688fef9fc1fa0e3ba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:26.958540 3398 log.cc:824] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:26.958585 3401 tablet_bootstrap.cc:654] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:26.959067 3401 log.cc:824] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:26.959336 3398 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: No bootstrap required, opened a new log
I20260808 00:41:26.959403 3398 ts_tablet_manager.cc:1397] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:41:26.959812 3401 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: No bootstrap required, opened a new log
I20260808 00:41:26.959949 3401 ts_tablet_manager.cc:1397] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 00:41:26.961133 3404 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: Bootstrap starting.
I20260808 00:41:26.961310 3398 raft_consensus.cc:348] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.961710 3398 raft_consensus.cc:374] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:26.961788 3398 raft_consensus.cc:729] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77397a8fbf9f4c3a83e2c15c05832a77, State: Initialized, Role: FOLLOWER
I20260808 00:41:26.961807 3404 tablet_bootstrap.cc:654] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:26.961947 3398 consensus_queue.cc:260] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.961905 3401 raft_consensus.cc:348] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.962108 3401 raft_consensus.cc:374] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:26.962163 3401 raft_consensus.cc:729] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9cd9774d89d493f89d2b51b86ef68d9, State: Initialized, Role: FOLLOWER
I20260808 00:41:26.962285 3398 ts_tablet_manager.cc:1428] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:41:26.962335 3132 heartbeater.cc:503] Master 127.1.208.62:33333 was elected leader, sending a full tablet report...
I20260808 00:41:26.962355 3401 consensus_queue.cc:260] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.962387 3404 log.cc:824] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:26.962653 3401 ts_tablet_manager.cc:1428] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:41:26.962666 3373 heartbeater.cc:503] Master 127.1.208.62:33333 was elected leader, sending a full tablet report...
I20260808 00:41:26.963548 3404 tablet_bootstrap.cc:492] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: No bootstrap required, opened a new log
I20260808 00:41:26.963614 3404 ts_tablet_manager.cc:1397] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:41:26.964916 3404 raft_consensus.cc:348] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.965039 3404 raft_consensus.cc:374] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:26.965060 3404 raft_consensus.cc:729] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ec10a811394c7baea22781b9cbb10c, State: Initialized, Role: FOLLOWER
I20260808 00:41:26.965163 3404 consensus_queue.cc:260] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:26.965431 3404 ts_tablet_manager.cc:1428] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 00:41:26.965484 2911 heartbeater.cc:503] Master 127.1.208.62:33333 was elected leader, sending a full tablet report...
W20260808 00:41:27.016851 3133 tablet.cc:2369] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:27.031734 3409 raft_consensus.cc:482] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:27.031873 3409 raft_consensus.cc:504] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:27.032214 3409 leader_election.cc:288] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9cd9774d89d493f89d2b51b86ef68d9 (127.1.208.3:33623), 77397a8fbf9f4c3a83e2c15c05832a77 (127.1.208.2:41481)
I20260808 00:41:27.035034 3396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.032470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:32883 (local address 127.1.208.2:41481)
0808 00:41:27.032534 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.032536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.032543 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:27.032573 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.032574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.032584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.032643 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.032829 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.033388 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034283 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.034699 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034763 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.034776 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.034927 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.034964 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:41:27.035043 3393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.032478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:57101 (local address 127.1.208.3:33623)
0808 00:41:27.032529 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.032531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.032655 (+ 124us) server_negotiation.cc:407] Connection header received
0808 00:41:27.032694 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.032695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.032706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.032774 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.032923 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.033417 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034346 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.034736 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034799 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.034809 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.034939 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.034977 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260808 00:41:27.035048 2919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.032379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:41481 (local address 127.1.208.1:32883)
0808 00:41:27.032417 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.032428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.032469 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.032660 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.032661 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.032663 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.032742 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.032744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.033463 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.033464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.034201 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.034203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.034778 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.034865 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.034869 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.034913 (+ 44us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.034957 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260808 00:41:27.035058 3410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.032379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:33623 (local address 127.1.208.1:57101)
0808 00:41:27.032594 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.032602 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.032650 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.032783 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.032784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.032786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.032864 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.032866 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.033499 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.033501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.034260 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.034263 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.034811 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.034812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.034878 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.034881 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.034926 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.034965 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":49,"thread_start_us":120,"threads_started":1}
I20260808 00:41:27.035396 3058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481f5cf306a4468688fef9fc1fa0e3ba" candidate_uuid: "91ec10a811394c7baea22781b9cbb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" is_pre_election: true
I20260808 00:41:27.035437 3287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481f5cf306a4468688fef9fc1fa0e3ba" candidate_uuid: "91ec10a811394c7baea22781b9cbb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" is_pre_election: true
I20260808 00:41:27.035561 3058 raft_consensus.cc:2454] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ec10a811394c7baea22781b9cbb10c in term 0.
I20260808 00:41:27.035569 3287 raft_consensus.cc:2454] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91ec10a811394c7baea22781b9cbb10c in term 0.
I20260808 00:41:27.035779 2712 leader_election.cc:302] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ec10a811394c7baea22781b9cbb10c, c9cd9774d89d493f89d2b51b86ef68d9; no voters:
I20260808 00:41:27.035965 3409 raft_consensus.cc:2781] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:27.036051 3409 raft_consensus.cc:482] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:27.036080 3409 raft_consensus.cc:3037] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.036994 3409 raft_consensus.cc:504] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:27.037137 3409 leader_election.cc:288] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [CANDIDATE]: Term 1 election: Requested vote from peers c9cd9774d89d493f89d2b51b86ef68d9 (127.1.208.3:33623), 77397a8fbf9f4c3a83e2c15c05832a77 (127.1.208.2:41481)
I20260808 00:41:27.037326 3287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481f5cf306a4468688fef9fc1fa0e3ba" candidate_uuid: "91ec10a811394c7baea22781b9cbb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9cd9774d89d493f89d2b51b86ef68d9"
I20260808 00:41:27.037354 3058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "481f5cf306a4468688fef9fc1fa0e3ba" candidate_uuid: "91ec10a811394c7baea22781b9cbb10c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77397a8fbf9f4c3a83e2c15c05832a77"
I20260808 00:41:27.037415 3287 raft_consensus.cc:3037] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.037457 3058 raft_consensus.cc:3037] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.038190 3287 raft_consensus.cc:2454] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ec10a811394c7baea22781b9cbb10c in term 1.
I20260808 00:41:27.038251 3058 raft_consensus.cc:2454] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91ec10a811394c7baea22781b9cbb10c in term 1.
I20260808 00:41:27.038367 2712 leader_election.cc:302] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ec10a811394c7baea22781b9cbb10c, c9cd9774d89d493f89d2b51b86ef68d9; no voters:
I20260808 00:41:27.038480 3409 raft_consensus.cc:2781] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:27.038633 3409 raft_consensus.cc:686] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [term 1 LEADER]: Becoming Leader. State: Replica: 91ec10a811394c7baea22781b9cbb10c, State: Running, Role: LEADER
I20260808 00:41:27.038733 3409 consensus_queue.cc:237] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } }
I20260808 00:41:27.039598 2563 catalog_manager.cc:5495] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c reported cstate change: term changed from 0 to 1, leader changed from <none> to 91ec10a811394c7baea22781b9cbb10c (127.1.208.1). New cstate: current_term: 1 leader_uuid: "91ec10a811394c7baea22781b9cbb10c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "91ec10a811394c7baea22781b9cbb10c" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 35015 } health_report { overall_health: HEALTHY } } }
I20260808 00:41:27.044714 3397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.042424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.1.208.1:35015)
0808 00:41:27.042467 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.042469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.042477 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:27.042547 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.042548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.042570 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.042631 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:27.042726 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.043236 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.044006 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.044197 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.044281 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.044417 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:27.044510 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:27.044582 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.044597 (+ 15us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.044613 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260808 00:41:27.046315 3287 raft_consensus.cc:1261] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 1 FOLLOWER]: Refusing update from remote peer 91ec10a811394c7baea22781b9cbb10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:27.046315 3058 raft_consensus.cc:1261] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 1 FOLLOWER]: Refusing update from remote peer 91ec10a811394c7baea22781b9cbb10c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:27.046653 3409 consensus_queue.cc:1035] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "77397a8fbf9f4c3a83e2c15c05832a77" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 41481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:27.046775 3413 consensus_queue.cc:1035] T 481f5cf306a4468688fef9fc1fa0e3ba P 91ec10a811394c7baea22781b9cbb10c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9cd9774d89d493f89d2b51b86ef68d9" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 33623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:27.048991 3418 mvcc.cc:205] Tried to move back new op lower bound from 7316069118139301888 to 7316069118111223808. Current Snapshot: MvccSnapshot[applied={T|T < 7316069118139301888}]
I20260808 00:41:27.049132 3420 mvcc.cc:205] Tried to move back new op lower bound from 7316069118139301888 to 7316069118111223808. Current Snapshot: MvccSnapshot[applied={T|T < 7316069118139301888}]
I20260808 00:41:27.049953 3422 mvcc.cc:205] Tried to move back new op lower bound from 7316069118139301888 to 7316069118111223808. Current Snapshot: MvccSnapshot[applied={T|T < 7316069118139301888 or (T in {7316069118139301888})}]
I20260808 00:41:27.051357 2563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:27.051436 2563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:27.052129 2563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c603c0f9cfe4c33bd6e4753fe1cfd3a: Sending DeleteTablet for 3 replicas of tablet 481f5cf306a4468688fef9fc1fa0e3ba
I20260808 00:41:27.052450 3024 tablet_service.cc:1514] Processing DeleteTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:32824
I20260808 00:41:27.052477 3247 tablet_service.cc:1514] Processing DeleteTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:50336
I20260808 00:41:27.052502 2797 tablet_service.cc:1514] Processing DeleteTablet for tablet 481f5cf306a4468688fef9fc1fa0e3ba with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='kudu', principal='kudu/127.1.208.62@KRBTEST.COM'} at 127.0.0.1:49420
I20260808 00:41:27.052703 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2681
I20260808 00:41:27.055440 3436 tablet_replica.cc:330] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: stopping tablet replica
I20260808 00:41:27.055544 3436 raft_consensus.cc:2229] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:27.055658 3436 raft_consensus.cc:2258] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:27.059252 3436 ts_tablet_manager.cc:1905] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:27.061440 3436 ts_tablet_manager.cc:1918] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:27.061578 3436 log.cc:1196] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/481f5cf306a4468688fef9fc1fa0e3ba
I20260808 00:41:27.061884 3436 ts_tablet_manager.cc:1939] T 481f5cf306a4468688fef9fc1fa0e3ba P c9cd9774d89d493f89d2b51b86ef68d9: Deleting consensus metadata
I20260808 00:41:27.062373 2540 catalog_manager.cc:4841] TS c9cd9774d89d493f89d2b51b86ef68d9 (127.1.208.3:33623): tablet 481f5cf306a4468688fef9fc1fa0e3ba (table test-table [id=039ed847a0b44cecbd2ed1d63d752b30]) successfully deleted
I20260808 00:41:27.065939 3434 tablet_replica.cc:330] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: stopping tablet replica
I20260808 00:41:27.066102 3434 raft_consensus.cc:2229] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:27.066277 3434 raft_consensus.cc:2258] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:27.067396 3434 ts_tablet_manager.cc:1905] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:27.068833 3434 ts_tablet_manager.cc:1918] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:27.068912 3434 log.cc:1196] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/481f5cf306a4468688fef9fc1fa0e3ba
I20260808 00:41:27.069178 3434 ts_tablet_manager.cc:1939] T 481f5cf306a4468688fef9fc1fa0e3ba P 77397a8fbf9f4c3a83e2c15c05832a77: Deleting consensus metadata
I20260808 00:41:27.069659 2537 catalog_manager.cc:4841] TS 77397a8fbf9f4c3a83e2c15c05832a77 (127.1.208.2:41481): tablet 481f5cf306a4468688fef9fc1fa0e3ba (table test-table [id=039ed847a0b44cecbd2ed1d63d752b30]) successfully deleted
W20260808 00:41:27.071174 2540 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:35015 (error 108)
I20260808 00:41:27.071475 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2924
I20260808 00:41:27.071653 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.071523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:35015 (local address 127.0.0.1:49446)
0808 00:41:27.071582 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.071619 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:41:27.079608 2540 catalog_manager.cc:4584] TS 91ec10a811394c7baea22781b9cbb10c (127.1.208.1:35015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 481f5cf306a4468688fef9fc1fa0e3ba: Network error: Client connection negotiation failed: client connection to 127.1.208.1:35015: connect: Connection refused (error 111)
I20260808 00:41:27.083715 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3141
I20260808 00:41:27.093884 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 2510
2026-08-08T00:41:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2103 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T00:41:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:27Z Disabled control of system clock
I20260808 00:41:27.124859 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40903
--webserver_interface=127.1.208.62
--webserver_port=0
--builtin_ntp_servers=127.1.208.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:27.205722 3455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:27.205950 3455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:27.205969 3455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:27.207118 3455 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:27.207159 3455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:27.207170 3455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:27.207193 3455 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:27.207214 3455 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:27.208561 3455 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40903
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:27.208794 3455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:27.209059 3455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:27.213598 3467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.213604 3468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.213660 3471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:27.213632 3455 server_base.cc:1034] running on GCE node
I20260808 00:41:27.214059 3455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:27.214298 3455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:27.215436 3455 hybrid_clock.cc:648] HybridClock initialized: now 1786149687215419 us; error 33 us; skew 500 ppm
I20260808 00:41:27.216863 3455 webserver.cc:460] Webserver started at http://127.1.208.62:46063/ using document root <none> and password file <none>
I20260808 00:41:27.217097 3455 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:27.217151 3455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:27.217284 3455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:27.218366 3455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "90586614fa384196a941be2a87d9d780"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "ad4bd1a75bc96827c4f92447ebf2fb24"
server_key_iv: "4a1bfd046f18d988573a0ad14bdc63f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.218727 3455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "90586614fa384196a941be2a87d9d780"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "ad4bd1a75bc96827c4f92447ebf2fb24"
server_key_iv: "4a1bfd046f18d988573a0ad14bdc63f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.219928 3455 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:27.220580 3480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.220739 3455 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:27.220836 3455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "90586614fa384196a941be2a87d9d780"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "ad4bd1a75bc96827c4f92447ebf2fb24"
server_key_iv: "4a1bfd046f18d988573a0ad14bdc63f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.220906 3455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:27.234215 3455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:27.234513 3455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:27.234639 3455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:27.238637 3455 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:40903
I20260808 00:41:27.238670 3571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:40903 every 8 connection(s)
I20260808 00:41:27.239019 3455 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:27.239966 3573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:27.242415 3573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: Bootstrap starting.
I20260808 00:41:27.242971 3573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:27.243256 3573 log.cc:824] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:27.243882 3573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: No bootstrap required, opened a new log
I20260808 00:41:27.245587 3573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } }
I20260808 00:41:27.245730 3573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:27.245764 3573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90586614fa384196a941be2a87d9d780, State: Initialized, Role: FOLLOWER
I20260808 00:41:27.245905 3573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } }
I20260808 00:41:27.245967 3573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:27.245987 3573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:27.246029 3573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.246519 3573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } }
I20260808 00:41:27.246611 3573 leader_election.cc:302] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90586614fa384196a941be2a87d9d780; no voters:
I20260808 00:41:27.246775 3573 leader_election.cc:288] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:27.246820 3579 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:27.246994 3579 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [term 1 LEADER]: Becoming Leader. State: Replica: 90586614fa384196a941be2a87d9d780, State: Running, Role: LEADER
I20260808 00:41:27.247036 3573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:27.247104 3579 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } }
I20260808 00:41:27.247521 3584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90586614fa384196a941be2a87d9d780. Latest consensus state: current_term: 1 leader_uuid: "90586614fa384196a941be2a87d9d780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } } }
I20260808 00:41:27.247617 3581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90586614fa384196a941be2a87d9d780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90586614fa384196a941be2a87d9d780" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40903 } } }
I20260808 00:41:27.247691 3581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:27.247905 3597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:27.247915 3584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:27.248454 3597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:27.249053 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3455
I20260808 00:41:27.249176 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:27.249434 1856 external_mini_cluster.cc:1428] Setting key 8761fb8d71e3420deed30e6dc1d8d10e
I20260808 00:41:27.250131 3597 catalog_manager.cc:1347] Generated new cluster ID: abafae75855246678aedae52b810bac3
I20260808 00:41:27.250185 3597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:27.258103 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.249922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45722 (local address 127.1.208.62:40903)
0808 00:41:27.253400 (+ 3478us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.253409 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.253443 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:41:27.253539 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.253545 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.253597 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.253720 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:27.253945 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.254870 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.255626 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.256451 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.256663 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.257440 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:27.257444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:27.257445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:27.257622 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:27.257660 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:27.257681 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:27.257744 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.257833 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.257914 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260808 00:41:27.259462 3597 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:27.259956 3597 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:27.263553 3597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: Generated new TSK 0
I20260808 00:41:27.263725 3597 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:27.266770 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.1:0
--local_ip_for_outbound_sockets=127.1.208.1
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--builtin_ntp_servers=127.1.208.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:27.372115 3611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:27.372326 3611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:27.372347 3611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:27.373539 3611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:27.373577 3611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:27.373601 3611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.1
I20260808 00:41:27.374930 3611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.208.1
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.1
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:27.375159 3611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:27.375460 3611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:27.381181 3621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.381198 3620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.381349 3623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:27.381527 3611 server_base.cc:1034] running on GCE node
I20260808 00:41:27.381711 3611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:27.381948 3611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:27.383085 3611 hybrid_clock.cc:648] HybridClock initialized: now 1786149687383072 us; error 30 us; skew 500 ppm
I20260808 00:41:27.384390 3611 webserver.cc:460] Webserver started at http://127.1.208.1:34657/ using document root <none> and password file <none>
I20260808 00:41:27.384686 3611 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:27.384750 3611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:27.384958 3611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:27.386236 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data/instance:
uuid: "55d7c91a258b41338e77251cdb0cf902"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "4de9c027ab7d809dd63a8c5ebdc385c2"
server_key_iv: "fff976cec9218073ae3f8918d4c7b7d7"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.386691 3611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance:
uuid: "55d7c91a258b41338e77251cdb0cf902"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "4de9c027ab7d809dd63a8c5ebdc385c2"
server_key_iv: "fff976cec9218073ae3f8918d4c7b7d7"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.388032 3611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:27.388792 3635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.388945 3611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:27.389031 3611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal
uuid: "55d7c91a258b41338e77251cdb0cf902"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "4de9c027ab7d809dd63a8c5ebdc385c2"
server_key_iv: "fff976cec9218073ae3f8918d4c7b7d7"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.389101 3611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:27.407249 3611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:27.407528 3611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:27.407650 3611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:27.407858 3611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:27.408165 3611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:27.408198 3611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.408237 3611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:27.408262 3611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.413923 3611 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.1:44017
I20260808 00:41:27.413988 3820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.1:44017 every 8 connection(s)
I20260808 00:41:27.414284 3611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/data/info.pb
I20260808 00:41:27.419013 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.414909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:56533 (local address 127.1.208.62:40903)
0808 00:41:27.414988 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.414991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.415225 (+ 234us) server_negotiation.cc:407] Connection header received
0808 00:41:27.415698 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.415699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.415712 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.415751 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:27.416407 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.416877 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.417877 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.418073 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.418123 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.418815 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:27.418817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:27.418817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:27.418822 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:27.418837 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:27.418840 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:27.418879 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.418932 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.418954 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 00:41:27.419198 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.414908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40903 (local address 127.1.208.1:56533)
0808 00:41:27.415107 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.415140 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.415617 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.415837 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.415841 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.415850 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:27.416342 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.416346 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.416947 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.416948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.417811 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.417814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.418133 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.418133 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.418240 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.418687 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:27.418687 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:27.418714 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:27.418746 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:27.418884 (+ 138us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:27.418886 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:27.418889 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.418932 (+ 43us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.419005 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":106,"threads_started":1}
I20260808 00:41:27.419726 3821 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40903
I20260808 00:41:27.419811 3821 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:27.419982 3821 heartbeater.cc:511] Master 127.1.208.62:40903 requested a full tablet report, sending...
I20260808 00:41:27.420449 3507 ts_manager.cc:194] Registered new tserver with Master: 55d7c91a258b41338e77251cdb0cf902 (127.1.208.1:44017)
I20260808 00:41:27.421025 3507 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.1:56533
I20260808 00:41:27.423820 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3611
I20260808 00:41:27.423918 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-0/wal/instance
I20260808 00:41:27.424098 1856 external_mini_cluster.cc:1428] Setting key 67c3ea0d8157aab7fc10a67497e9afe8
I20260808 00:41:27.424973 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.2:0
--local_ip_for_outbound_sockets=127.1.208.2
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--builtin_ntp_servers=127.1.208.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:27.506871 3824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:27.507069 3824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:27.507089 3824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:27.508221 3824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:27.508260 3824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:27.508284 3824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.2
I20260808 00:41:27.509639 3824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.208.2
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.2
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:27.509850 3824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:27.510120 3824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:27.514882 3835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.514870 3832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.514856 3831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:27.515107 3824 server_base.cc:1034] running on GCE node
I20260808 00:41:27.515345 3824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:27.515544 3824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:27.516680 3824 hybrid_clock.cc:648] HybridClock initialized: now 1786149687516666 us; error 29 us; skew 500 ppm
I20260808 00:41:27.517933 3824 webserver.cc:460] Webserver started at http://127.1.208.2:39823/ using document root <none> and password file <none>
I20260808 00:41:27.518149 3824 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:27.518198 3824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:27.518383 3824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:27.519424 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data/instance:
uuid: "03d61953376b447cae649120de53e40b"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "d28591e6ab8cb9d780bb3fe0e0da8be6"
server_key_iv: "8eb526642e594d102a82cc9e3faeb9f9"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.519759 3824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance:
uuid: "03d61953376b447cae649120de53e40b"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "d28591e6ab8cb9d780bb3fe0e0da8be6"
server_key_iv: "8eb526642e594d102a82cc9e3faeb9f9"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.520952 3824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:27.521646 3844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.521786 3824 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.521875 3824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal
uuid: "03d61953376b447cae649120de53e40b"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "d28591e6ab8cb9d780bb3fe0e0da8be6"
server_key_iv: "8eb526642e594d102a82cc9e3faeb9f9"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.521937 3824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:27.534467 3824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:27.534708 3824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:27.534825 3824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:27.535014 3824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:27.535315 3824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:27.535348 3824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.535374 3824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:27.535398 3824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.541013 3824 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.2:43047
I20260808 00:41:27.541078 4048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.2:43047 every 8 connection(s)
I20260808 00:41:27.541388 3824 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/data/info.pb
I20260808 00:41:27.546190 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.542018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.2:44457 (local address 127.1.208.62:40903)
0808 00:41:27.542096 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.542099 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.542297 (+ 198us) server_negotiation.cc:407] Connection header received
0808 00:41:27.542748 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.542748 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.542758 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.542807 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:27.543428 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.543920 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.545003 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.545195 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.545244 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.545974 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:27.545975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:27.545976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:27.545980 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:27.545993 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:27.545996 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:27.546036 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.546099 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.546128 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 00:41:27.546396 4053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.542005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40903 (local address 127.1.208.2:44457)
0808 00:41:27.542183 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.542216 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.542676 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.542860 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.542863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.542872 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:27.543363 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.543369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.543991 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.543992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.544924 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.544928 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.545256 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.545257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.545414 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.545845 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:27.545845 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:27.545875 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:27.545897 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:27.546043 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:27.546045 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:27.546047 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.546092 (+ 45us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.546167 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260808 00:41:27.546864 4050 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40903
I20260808 00:41:27.546954 4050 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:27.547127 4050 heartbeater.cc:511] Master 127.1.208.62:40903 requested a full tablet report, sending...
I20260808 00:41:27.547505 3507 ts_manager.cc:194] Registered new tserver with Master: 03d61953376b447cae649120de53e40b (127.1.208.2:43047)
I20260808 00:41:27.547914 3507 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.2:44457
I20260808 00:41:27.550333 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 3824
I20260808 00:41:27.550431 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal/instance
I20260808 00:41:27.550603 1856 external_mini_cluster.cc:1428] Setting key f8afbbcc81a693fdaa9115cacaf0a1cc
I20260808 00:41:27.551492 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.208.3:0
--local_ip_for_outbound_sockets=127.1.208.3
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--builtin_ntp_servers=127.1.208.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 00:41:27.636706 4056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:27.636907 4056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:27.636940 4056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:27.638091 4056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:27.638137 4056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:27.638172 4056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.208.3
I20260808 00:41:27.639499 4056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.208.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.208.3
--webserver_port=0
--tserver_master_addrs=127.1.208.62:40903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.208.3
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:27.639758 4056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:27.640051 4056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:27.644994 4069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:27.645078 4067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:27.645087 4056 server_base.cc:1034] running on GCE node
W20260808 00:41:27.644960 4074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:27.645464 4056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:27.645707 4056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:27.646867 4056 hybrid_clock.cc:648] HybridClock initialized: now 1786149687646849 us; error 35 us; skew 500 ppm
I20260808 00:41:27.648177 4056 webserver.cc:460] Webserver started at http://127.1.208.3:44755/ using document root <none> and password file <none>
I20260808 00:41:27.648413 4056 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:27.648458 4056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:27.648568 4056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:27.649662 4056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data/instance:
uuid: "7c55aaecdc734fe19d919f6685d35d1a"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "e3149716c35343e12c24f5a4543dd7bc"
server_key_iv: "6acea76705c7d104cf8e83feae3526f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.649982 4056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance:
uuid: "7c55aaecdc734fe19d919f6685d35d1a"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "e3149716c35343e12c24f5a4543dd7bc"
server_key_iv: "6acea76705c7d104cf8e83feae3526f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.651185 4056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:27.651835 4084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.651976 4056 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:27.652057 4056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal
uuid: "7c55aaecdc734fe19d919f6685d35d1a"
format_stamp: "Formatted at 2026-08-08 00:41:27 on dist-test-slave-gjw7"
server_key: "e3149716c35343e12c24f5a4543dd7bc"
server_key_iv: "6acea76705c7d104cf8e83feae3526f1"
server_key_version: "encryptionkey@0"
I20260808 00:41:27.652119 4056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:27.660269 4056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:27.660519 4056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:27.660636 4056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:27.660830 4056 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 00:41:27.661128 4056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 00:41:27.661163 4056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.661190 4056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 00:41:27.661214 4056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:27.666719 4056 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.3:43131
I20260808 00:41:27.666785 4280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.3:43131 every 8 connection(s)
I20260808 00:41:27.667068 4056 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/data/info.pb
I20260808 00:41:27.671963 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.667768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:41117 (local address 127.1.208.62:40903)
0808 00:41:27.667818 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.667820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.668015 (+ 195us) server_negotiation.cc:407] Connection header received
0808 00:41:27.668482 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.668483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.668496 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.668552 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:27.669182 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.669785 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.670713 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.670907 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.670962 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.671726 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:27.671728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:27.671728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:27.671734 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:27.671749 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:27.671753 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:27.671804 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.671875 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.671903 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260808 00:41:27.672139 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.667729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:40903 (local address 127.1.208.3:41117)
0808 00:41:27.667909 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.667947 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.668402 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.668607 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.668610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.668618 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:27.669099 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.669105 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.669862 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.669862 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.670623 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.670626 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.670969 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.670970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.671072 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.671574 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 00:41:27.671574 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:27.671602 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 00:41:27.671625 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:27.671810 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:27.671813 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:27.671815 (+ 2us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.671866 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.671942 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260808 00:41:27.672612 4281 heartbeater.cc:347] Connected to a master server at 127.1.208.62:40903
I20260808 00:41:27.672696 4281 heartbeater.cc:464] Registering TS with master...
I20260808 00:41:27.672880 4281 heartbeater.cc:511] Master 127.1.208.62:40903 requested a full tablet report, sending...
I20260808 00:41:27.673249 3507 ts_manager.cc:194] Registered new tserver with Master: 7c55aaecdc734fe19d919f6685d35d1a (127.1.208.3:43131)
I20260808 00:41:27.673663 3507 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.208.3:41117
I20260808 00:41:27.677208 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4056
I20260808 00:41:27.677335 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal/instance
I20260808 00:41:27.677516 1856 external_mini_cluster.cc:1428] Setting key c93ebd3ce97969cb060edf8e7e17fd96
I20260808 00:41:27.678457 1856 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 00:41:27.682391 3610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.679760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45726 (local address 127.1.208.62:40903)
0808 00:41:27.679834 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.679836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.679875 (+ 39us) server_negotiation.cc:407] Connection header received
0808 00:41:27.679934 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.679935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.679943 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.679978 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:27.680119 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.680574 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.681261 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.681484 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.681533 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.682131 (+ 598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:27.682132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:27.682132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:27.682136 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:27.682149 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:27.682152 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:27.682208 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.682296 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.682320 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 00:41:27.683755 3507 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 00:41:27.685272 3507 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 00:41:27.691497 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:43131 (local address 127.0.0.1:32882)
0808 00:41:27.687692 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.687701 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.687752 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.688307 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.688308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.688310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.688351 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.688352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689530 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.689531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.690440 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.690443 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691193 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.691270 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691285 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.691369 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.691419 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
I20260808 00:41:27.691501 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32882 (local address 127.1.208.3:43131)
0808 00:41:27.688163 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.688168 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.688177 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:27.688214 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.688215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.688229 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.688309 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.688414 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689459 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.690515 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691116 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691187 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691273 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.691389 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.691422 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":78,"threads_started":1}
I20260808 00:41:27.691711 4301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43047 (local address 127.0.0.1:57750)
0808 00:41:27.687898 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.687906 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.687947 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.688451 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.688452 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.688454 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.688482 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.688483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689330 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.689331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.690692 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.690695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691527 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.691584 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691588 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.691618 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.691654 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":54,"thread_start_us":60,"threads_started":1}
I20260808 00:41:27.691790 4292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44017 (local address 127.0.0.1:59554)
0808 00:41:27.687644 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.687687 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.687752 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.688050 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.688054 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.688056 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.688126 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.688128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689429 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.689430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.690692 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.690695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691569 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.691630 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691635 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.691663 (+ 28us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.691694 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":87,"thread_start_us":85,"threads_started":1}
I20260808 00:41:27.692086 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57750 (local address 127.1.208.2:43047)
0808 00:41:27.688273 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.688277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.688286 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:27.688333 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.688337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.688352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.688455 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.688521 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689231 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.690790 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691440 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691773 (+ 333us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691884 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.691910 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.691952 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":114,"threads_started":1}
I20260808 00:41:27.692214 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.687527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59554 (local address 127.1.208.1:44017)
0808 00:41:27.687892 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.687896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.687906 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:27.687944 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.687947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.687962 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.688042 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.688205 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.689357 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.690775 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.691471 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.691857 (+ 386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.691932 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.691958 (+ 26us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.692133 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":81,"threads_started":1}
I20260808 00:41:27.692862 4167 tablet_service.cc:1467] Processing CreateTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a (DEFAULT_TABLE table=test-table [id=c7e3fd40397b46b992054d0923c58c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:27.692893 3719 tablet_service.cc:1467] Processing CreateTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a (DEFAULT_TABLE table=test-table [id=c7e3fd40397b46b992054d0923c58c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:27.693197 3719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82f12cd73984db2a53a3e7892bbaa2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:27.693222 4167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82f12cd73984db2a53a3e7892bbaa2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:27.695789 4306 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: Bootstrap starting.
I20260808 00:41:27.696009 4304 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Bootstrap starting.
I20260808 00:41:27.696192 3940 tablet_service.cc:1467] Processing CreateTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a (DEFAULT_TABLE table=test-table [id=c7e3fd40397b46b992054d0923c58c64]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 00:41:27.696554 3940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a82f12cd73984db2a53a3e7892bbaa2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:27.696565 4306 tablet_bootstrap.cc:654] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:27.696591 4304 tablet_bootstrap.cc:654] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:27.696863 4304 log.cc:824] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:27.696897 4306 log.cc:824] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:27.697621 4306 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: No bootstrap required, opened a new log
I20260808 00:41:27.697639 4304 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: No bootstrap required, opened a new log
I20260808 00:41:27.697686 4306 ts_tablet_manager.cc:1397] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:27.697698 4304 ts_tablet_manager.cc:1397] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:27.699618 4306 raft_consensus.cc:348] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.699754 4306 raft_consensus.cc:374] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:27.699786 4306 raft_consensus.cc:729] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d7c91a258b41338e77251cdb0cf902, State: Initialized, Role: FOLLOWER
I20260808 00:41:27.699961 4306 consensus_queue.cc:260] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.700027 4311 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Bootstrap starting.
I20260808 00:41:27.700273 4306 ts_tablet_manager.cc:1428] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 00:41:27.700174 4304 raft_consensus.cc:348] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.700306 3821 heartbeater.cc:503] Master 127.1.208.62:40903 was elected leader, sending a full tablet report...
I20260808 00:41:27.700387 4304 raft_consensus.cc:374] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:27.700425 4304 raft_consensus.cc:729] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c55aaecdc734fe19d919f6685d35d1a, State: Initialized, Role: FOLLOWER
I20260808 00:41:27.700533 4304 consensus_queue.cc:260] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.700668 4311 tablet_bootstrap.cc:654] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:27.700788 4304 ts_tablet_manager.cc:1428] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 00:41:27.700824 4281 heartbeater.cc:503] Master 127.1.208.62:40903 was elected leader, sending a full tablet report...
I20260808 00:41:27.701041 4311 log.cc:824] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:27.701846 4311 tablet_bootstrap.cc:492] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: No bootstrap required, opened a new log
I20260808 00:41:27.701901 4311 ts_tablet_manager.cc:1397] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 00:41:27.703727 4311 raft_consensus.cc:348] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.703851 4311 raft_consensus.cc:374] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:27.703883 4311 raft_consensus.cc:729] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03d61953376b447cae649120de53e40b, State: Initialized, Role: FOLLOWER
I20260808 00:41:27.703991 4311 consensus_queue.cc:260] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.704201 4311 ts_tablet_manager.cc:1428] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 00:41:27.704267 4050 heartbeater.cc:503] Master 127.1.208.62:40903 was elected leader, sending a full tablet report...
W20260808 00:41:27.791936 4052 tablet.cc:2369] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:27.801338 4312 raft_consensus.cc:482] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:27.801460 4312 raft_consensus.cc:504] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.801771 4312 leader_election.cc:288] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c55aaecdc734fe19d919f6685d35d1a (127.1.208.3:43131), 03d61953376b447cae649120de53e40b (127.1.208.2:43047)
I20260808 00:41:27.803089 4314 raft_consensus.cc:482] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 00:41:27.803241 4314 raft_consensus.cc:504] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.803582 4314 leader_election.cc:288] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55d7c91a258b41338e77251cdb0cf902 (127.1.208.1:44017), 03d61953376b447cae649120de53e40b (127.1.208.2:43047)
I20260808 00:41:27.805284 4302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.802171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:38789 (local address 127.1.208.2:43047)
0808 00:41:27.802219 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.802221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.802361 (+ 140us) server_negotiation.cc:407] Connection header received
0808 00:41:27.802396 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.802397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.802406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.802451 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.802595 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.803082 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.804343 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.804908 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.804953 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.804962 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.805174 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.805211 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":24}
I20260808 00:41:27.805565 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.801970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.3:43131 (local address 127.1.208.1:34383)
0808 00:41:27.802021 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.802033 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.802088 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.802212 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.802213 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.802214 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.802291 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.802293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.802915 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.802916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.804094 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.804097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805316 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.805318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.805389 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.805393 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.805440 (+ 47us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.805481 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 00:41:27.805652 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.802038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.1:34383 (local address 127.1.208.3:43131)
0808 00:41:27.802099 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.802100 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.802109 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:27.802141 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.802142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.802150 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.802201 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.802369 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.802842 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.804243 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.804798 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.804872 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.804883 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.805557 (+ 674us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.805593 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:41:27.805706 4319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.802081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43047 (local address 127.1.208.1:38789)
0808 00:41:27.802275 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.802283 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.802338 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.802460 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.802461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.802462 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.802537 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.802538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.803168 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.803170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.804253 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.804255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805030 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.805031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.805103 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.805107 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.805390 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.805433 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":132,"threads_started":1}
I20260808 00:41:27.805927 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "55d7c91a258b41338e77251cdb0cf902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c55aaecdc734fe19d919f6685d35d1a" is_pre_election: true
I20260808 00:41:27.806030 4212 raft_consensus.cc:2454] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55d7c91a258b41338e77251cdb0cf902 in term 0.
I20260808 00:41:27.806041 3974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "55d7c91a258b41338e77251cdb0cf902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03d61953376b447cae649120de53e40b" is_pre_election: true
I20260808 00:41:27.806188 3974 raft_consensus.cc:2454] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55d7c91a258b41338e77251cdb0cf902 in term 0.
I20260808 00:41:27.806442 3636 leader_election.cc:302] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55d7c91a258b41338e77251cdb0cf902, 7c55aaecdc734fe19d919f6685d35d1a; no voters:
I20260808 00:41:27.806629 4312 raft_consensus.cc:2781] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 00:41:27.806710 4312 raft_consensus.cc:482] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 00:41:27.806731 4312 raft_consensus.cc:3037] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.807122 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.803798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:47509 (local address 127.1.208.1:44017)
0808 00:41:27.803834 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.803836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.804078 (+ 242us) server_negotiation.cc:407] Connection header received
0808 00:41:27.804172 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.804173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.804182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.804210 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.804429 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805046 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.806391 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.806819 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.806885 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.806894 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.807035 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.807069 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:27.807138 4321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.803718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44017 (local address 127.1.208.3:47509)
0808 00:41:27.804016 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.804025 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.804103 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.804266 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.804267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.804269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.804342 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.804343 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805130 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.805130 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.806290 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.806293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.806889 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.806890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.806961 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.806965 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.807019 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.807055 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":28}
I20260808 00:41:27.807453 3747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "7c55aaecdc734fe19d919f6685d35d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55d7c91a258b41338e77251cdb0cf902" is_pre_election: true
I20260808 00:41:27.807689 4312 raft_consensus.cc:504] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.807823 3747 raft_consensus.cc:2379] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7c55aaecdc734fe19d919f6685d35d1a in current term 1: Already voted for candidate 55d7c91a258b41338e77251cdb0cf902 in this term.
I20260808 00:41:27.807829 4312 leader_election.cc:288] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [CANDIDATE]: Term 1 election: Requested vote from peers 7c55aaecdc734fe19d919f6685d35d1a (127.1.208.3:43131), 03d61953376b447cae649120de53e40b (127.1.208.2:43047)
I20260808 00:41:27.808015 3974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "55d7c91a258b41338e77251cdb0cf902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03d61953376b447cae649120de53e40b"
I20260808 00:41:27.808070 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "55d7c91a258b41338e77251cdb0cf902" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c55aaecdc734fe19d919f6685d35d1a"
I20260808 00:41:27.808092 3974 raft_consensus.cc:3037] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.808143 4212 raft_consensus.cc:3037] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:27.808250 4284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.803843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.2:43047 (local address 127.1.208.3:33215)
0808 00:41:27.804193 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.804204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:27.804248 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:27.804433 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:27.804433 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:27.804435 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 00:41:27.804494 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.804496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805362 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.805363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.807049 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:27.807051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.808048 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.808049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:27.808112 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.808116 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 00:41:27.808146 (+ 30us) client_negotiation.cc:240] Negotiation successful
0808 00:41:27.808185 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":23,"thread_start_us":112,"threads_started":1}
I20260808 00:41:27.808522 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.804048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.208.3:33215 (local address 127.1.208.2:43047)
0808 00:41:27.804199 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.804200 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.804300 (+ 100us) server_negotiation.cc:407] Connection header received
0808 00:41:27.804336 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.804337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.804346 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.804549 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 00:41:27.804578 (+ 29us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.805237 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.807130 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.807942 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.808362 (+ 420us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.808373 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.808422 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.808457 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260808 00:41:27.808709 3973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a82f12cd73984db2a53a3e7892bbaa2a" candidate_uuid: "7c55aaecdc734fe19d919f6685d35d1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03d61953376b447cae649120de53e40b" is_pre_election: true
I20260808 00:41:27.808956 4212 raft_consensus.cc:2454] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55d7c91a258b41338e77251cdb0cf902 in term 1.
I20260808 00:41:27.809028 4085 leader_election.cc:302] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7c55aaecdc734fe19d919f6685d35d1a; no voters: 03d61953376b447cae649120de53e40b, 55d7c91a258b41338e77251cdb0cf902
I20260808 00:41:27.809048 3974 raft_consensus.cc:2454] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55d7c91a258b41338e77251cdb0cf902 in term 1.
I20260808 00:41:27.809178 3636 leader_election.cc:302] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 03d61953376b447cae649120de53e40b, 55d7c91a258b41338e77251cdb0cf902; no voters:
I20260808 00:41:27.809227 4314 raft_consensus.cc:2726] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 00:41:27.809307 4312 raft_consensus.cc:2781] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:27.809470 4312 raft_consensus.cc:686] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [term 1 LEADER]: Becoming Leader. State: Replica: 55d7c91a258b41338e77251cdb0cf902, State: Running, Role: LEADER
I20260808 00:41:27.809551 4312 consensus_queue.cc:237] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } }
I20260808 00:41:27.810400 3507 catalog_manager.cc:5495] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 reported cstate change: term changed from 0 to 1, leader changed from <none> to 55d7c91a258b41338e77251cdb0cf902 (127.1.208.1). New cstate: current_term: 1 leader_uuid: "55d7c91a258b41338e77251cdb0cf902" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7c91a258b41338e77251cdb0cf902" member_type: VOTER last_known_addr { host: "127.1.208.1" port: 44017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 } health_report { overall_health: UNKNOWN } } }
I20260808 00:41:27.838186 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.835904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59560 (local address 127.1.208.1:44017)
0808 00:41:27.835972 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:27.835974 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:27.835983 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:27.836027 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:27.836028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:27.836045 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:27.836105 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:27.836241 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.836718 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.837479 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:27.837662 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:27.837715 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:27.837870 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:27.837975 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:27.838048 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:27.838069 (+ 21us) server_negotiation.cc:299] Negotiation successful
0808 00:41:27.838088 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 00:41:27.840001 3974 raft_consensus.cc:1261] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 1 FOLLOWER]: Refusing update from remote peer 55d7c91a258b41338e77251cdb0cf902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:27.840039 4212 raft_consensus.cc:1261] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 1 FOLLOWER]: Refusing update from remote peer 55d7c91a258b41338e77251cdb0cf902: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:27.840361 4312 consensus_queue.cc:1035] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c55aaecdc734fe19d919f6685d35d1a" member_type: VOTER last_known_addr { host: "127.1.208.3" port: 43131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:27.840510 4324 consensus_queue.cc:1035] T a82f12cd73984db2a53a3e7892bbaa2a P 55d7c91a258b41338e77251cdb0cf902 [LEADER]: Connected to new peer: Peer: permanent_uuid: "03d61953376b447cae649120de53e40b" member_type: VOTER last_known_addr { host: "127.1.208.2" port: 43047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:27.843084 4336 mvcc.cc:205] Tried to move back new op lower bound from 7316069121390092288 to 7316069121268445184. Current Snapshot: MvccSnapshot[applied={T|T < 7316069121390092288}]
I20260808 00:41:27.844882 3507 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:27.844965 3507 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 00:41:27.845719 3507 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 90586614fa384196a941be2a87d9d780: Sending DeleteTablet for 3 replicas of tablet a82f12cd73984db2a53a3e7892bbaa2a
I20260808 00:41:27.846035 3719 tablet_service.cc:1514] Processing DeleteTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='slave'} at 127.0.0.1:59554
I20260808 00:41:27.846241 4167 tablet_service.cc:1514] Processing DeleteTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='slave'} at 127.0.0.1:32882
I20260808 00:41:27.846344 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3611
I20260808 00:41:27.846451 3940 tablet_service.cc:1514] Processing DeleteTablet for tablet a82f12cd73984db2a53a3e7892bbaa2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 00:41:27 UTC) from {username='slave'} at 127.0.0.1:57750
I20260808 00:41:27.846742 4354 tablet_replica.cc:330] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: stopping tablet replica
I20260808 00:41:27.849174 4354 raft_consensus.cc:2229] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:27.849360 4354 raft_consensus.cc:2258] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:27.853116 4354 ts_tablet_manager.cc:1905] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:27.854940 4354 ts_tablet_manager.cc:1918] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:27.855062 4354 log.cc:1196] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-2/wal/wals/a82f12cd73984db2a53a3e7892bbaa2a
I20260808 00:41:27.855394 4354 ts_tablet_manager.cc:1939] T a82f12cd73984db2a53a3e7892bbaa2a P 7c55aaecdc734fe19d919f6685d35d1a: Deleting consensus metadata
I20260808 00:41:27.855849 3481 catalog_manager.cc:4841] TS 7c55aaecdc734fe19d919f6685d35d1a (127.1.208.3:43131): tablet a82f12cd73984db2a53a3e7892bbaa2a (table test-table [id=c7e3fd40397b46b992054d0923c58c64]) successfully deleted
I20260808 00:41:27.857416 4355 tablet_replica.cc:330] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: stopping tablet replica
I20260808 00:41:27.857527 4355 raft_consensus.cc:2229] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 00:41:27.857633 4355 raft_consensus.cc:2258] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 00:41:27.857988 4355 ts_tablet_manager.cc:1905] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 00:41:27.859251 4355 ts_tablet_manager.cc:1918] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 00:41:27.859315 4355 log.cc:1196] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Deleting WAL directory at /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786149440113891-1856-0/minicluster-data/ts-1/wal/wals/a82f12cd73984db2a53a3e7892bbaa2a
I20260808 00:41:27.859560 4355 ts_tablet_manager.cc:1939] T a82f12cd73984db2a53a3e7892bbaa2a P 03d61953376b447cae649120de53e40b: Deleting consensus metadata
I20260808 00:41:27.859925 3481 catalog_manager.cc:4841] TS 03d61953376b447cae649120de53e40b (127.1.208.2:43047): tablet a82f12cd73984db2a53a3e7892bbaa2a (table test-table [id=c7e3fd40397b46b992054d0923c58c64]) successfully deleted
W20260808 00:41:27.861594 3484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.208.1:44017 (error 108)
I20260808 00:41:27.861845 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3824
I20260808 00:41:27.863947 4291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:27.861930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.1:44017 (local address 127.0.0.1:59574)
0808 00:41:27.863865 (+ 1935us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:27.863914 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44017: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1912}
W20260808 00:41:27.868049 3484 catalog_manager.cc:4584] TS 55d7c91a258b41338e77251cdb0cf902 (127.1.208.1:44017): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a82f12cd73984db2a53a3e7892bbaa2a: Network error: Client connection negotiation failed: client connection to 127.1.208.1:44017: connect: Connection refused (error 111)
I20260808 00:41:27.873414 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4056
I20260808 00:41:27.880143 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 3455
2026-08-08T00:41:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (786 ms)
[----------] 6 tests from EncryptionPolicyTest (9238 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:27 dist-test-slave-gjw7 krb5kdc[4359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:27 dist-test-slave-gjw7 krb5kdc[4359](info): set up 2 sockets
Aug 08 00:41:27 dist-test-slave-gjw7 krb5kdc[4359](info): commencing operation
krb5kdc: starting...
W20260808 00:41:29.048774 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:29.165673 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46399
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46399
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:29.246830 4383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:29.247030 4383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:29.247051 4383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:29.248128 4383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:29.248162 4383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:29.248174 4383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:29.248185 4383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:29.248195 4383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:29.249636 4383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46399
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:29.249871 4383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:29.250059 4383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:29.254670 4394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:29.254676 4398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:29.254745 4383 server_base.cc:1034] running on GCE node
W20260808 00:41:29.254716 4396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:29.255208 4383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:29.255503 4383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:29.256676 4383 hybrid_clock.cc:648] HybridClock initialized: now 1786149689256655 us; error 37 us; skew 500 ppm
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:29.259150 4383 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:29.259595 4383 webserver.cc:460] Webserver started at http://127.0.0.1:41545/ using document root <none> and password file <none>
I20260808 00:41:29.259828 4383 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:29.259882 4383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:29.260017 4383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:29.261087 4383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "26324ace75c34ee1915599b31fdaa208"
format_stamp: "Formatted at 2026-08-08 00:41:29 on dist-test-slave-gjw7"
server_key: "dd51e36ff5500736991c5808e114058d"
server_key_iv: "75c7224f09b6bbf6116d3f26988e7fa9"
server_key_version: "encryptionkey@0"
I20260808 00:41:29.261469 4383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "26324ace75c34ee1915599b31fdaa208"
format_stamp: "Formatted at 2026-08-08 00:41:29 on dist-test-slave-gjw7"
server_key: "dd51e36ff5500736991c5808e114058d"
server_key_iv: "75c7224f09b6bbf6116d3f26988e7fa9"
server_key_version: "encryptionkey@0"
I20260808 00:41:29.262694 4383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:41:29.263360 4407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:29.263514 4383 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:29.263590 4383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "26324ace75c34ee1915599b31fdaa208"
format_stamp: "Formatted at 2026-08-08 00:41:29 on dist-test-slave-gjw7"
server_key: "dd51e36ff5500736991c5808e114058d"
server_key_iv: "75c7224f09b6bbf6116d3f26988e7fa9"
server_key_version: "encryptionkey@0"
I20260808 00:41:29.263656 4383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:29.299727 4383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:29.301537 4383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:29.301676 4383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:29.305639 4383 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46399
I20260808 00:41:29.305672 4503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46399 every 8 connection(s)
I20260808 00:41:29.306143 4383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:29.307195 4504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:29.309830 4504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208: Bootstrap starting.
I20260808 00:41:29.310384 4504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:29.310678 4504 log.cc:824] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:29.310691 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4383
I20260808 00:41:29.310792 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:29.310983 1856 external_mini_cluster.cc:1428] Setting key f77bc945df7a2d1cb3367222cb3e2fa7
I20260808 00:41:29.311427 4504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208: No bootstrap required, opened a new log
I20260808 00:41:29.313123 4504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } }
I20260808 00:41:29.313279 4504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:29.313376 4504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26324ace75c34ee1915599b31fdaa208, State: Initialized, Role: FOLLOWER
I20260808 00:41:29.313519 4504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } }
I20260808 00:41:29.313604 4504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:29.313642 4504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:29.313688 4504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:29.314553 4504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } }
I20260808 00:41:29.314652 4504 leader_election.cc:302] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26324ace75c34ee1915599b31fdaa208; no voters:
I20260808 00:41:29.314811 4504 leader_election.cc:288] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:29.314843 4513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:29.314947 4513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [term 1 LEADER]: Becoming Leader. State: Replica: 26324ace75c34ee1915599b31fdaa208, State: Running, Role: LEADER
I20260808 00:41:29.315081 4504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:29.315099 4513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } }
I20260808 00:41:29.315491 4513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26324ace75c34ee1915599b31fdaa208. Latest consensus state: current_term: 1 leader_uuid: "26324ace75c34ee1915599b31fdaa208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } } }
I20260808 00:41:29.315595 4513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:29.315752 4515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26324ace75c34ee1915599b31fdaa208" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26324ace75c34ee1915599b31fdaa208" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46399 } } }
I20260808 00:41:29.315959 4515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:29.316468 4528 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:29.317046 4528 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:41:29.319098 4528 catalog_manager.cc:1347] Generated new cluster ID: e0d6ca01a5474234b42db5afc35d194a
I20260808 00:41:29.319152 4528 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:29.322414 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:29.311498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33841 (local address 127.0.0.1:46399)
0808 00:41:29.311672 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:29.311680 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:29.311706 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:41:29.313105 (+ 1399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:29.313113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:29.313168 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:29.313332 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:29.314102 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:29.315135 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:29.316173 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:29.316707 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:29.316810 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:29.319445 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:29.319448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:29.319449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:29.319461 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:29.320953 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:29.321344 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:29.321347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:29.321347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:29.321448 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:29.321652 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:29.321652 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:29.321653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:29.321878 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:29.322038 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:29.322120 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:41:29.322218 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260808 00:41:29.323504 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:29.330163 4511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:29.324751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43039 (local address 127.0.0.1:46399)
0808 00:41:29.324795 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:29.324796 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:29.324902 (+ 106us) server_negotiation.cc:407] Connection header received
0808 00:41:29.324953 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:29.324954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:29.324964 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:29.325006 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:29.325519 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:29.325979 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:29.326617 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:29.326831 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:29.326884 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:29.327882 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:29.327883 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:29.327883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:29.327890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:29.329090 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:29.329469 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:29.329470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:29.329470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:29.329523 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:29.329707 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:29.329708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:29.329708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:29.329809 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:29.329895 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:29.330027 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 00:41:29.330086 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260808 00:41:29.333065 4528 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:29.333760 4528 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:29.344705 4528 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26324ace75c34ee1915599b31fdaa208: Generated new TSK 0
I20260808 00:41:29.344887 4528 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:29.347422 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4383
W20260808 00:41:29.363607 4357 connection.cc:535] client connection to 127.0.0.1:46399 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46399): Connection reset by peer (error 104)
2026-08-08T00:41:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4574](info): set up 2 sockets
Aug 08 00:41:29 dist-test-slave-gjw7 krb5kdc[4574](info): commencing operation
krb5kdc: starting...
W20260808 00:41:30.545852 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:30 dist-test-slave-gjw7 krb5kdc[4574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:30.664510 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:30.745841 4596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:30.746053 4596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:30.746083 4596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:30.747222 4596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:30.747267 4596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:30.747287 4596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:30.747303 4596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:30.747318 4596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:30.748796 4596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:30.749044 4596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:30.749257 4596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:30.753829 4603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:30.753872 4596 server_base.cc:1034] running on GCE node
W20260808 00:41:30.753837 4608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:30.753888 4602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:30.754211 4596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:30.754456 4596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:30.755595 4596 hybrid_clock.cc:648] HybridClock initialized: now 1786149690755575 us; error 33 us; skew 500 ppm
Aug 08 00:41:30 dist-test-slave-gjw7 krb5kdc[4574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149690, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:30.762609 4596 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:30.763055 4596 webserver.cc:460] Webserver started at http://127.0.0.1:44691/ using document root <none> and password file <none>
I20260808 00:41:30.763270 4596 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:30.763324 4596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:30.763460 4596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:30.764556 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "0ce38fc80f734d85aa7d8ee645e74ffb"
format_stamp: "Formatted at 2026-08-08 00:41:30 on dist-test-slave-gjw7"
server_key: "2055500807fe991f3be474a479784127"
server_key_iv: "1765c4a392380d97eb6313af320fc86b"
server_key_version: "encryptionkey@0"
I20260808 00:41:30.764906 4596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "0ce38fc80f734d85aa7d8ee645e74ffb"
format_stamp: "Formatted at 2026-08-08 00:41:30 on dist-test-slave-gjw7"
server_key: "2055500807fe991f3be474a479784127"
server_key_iv: "1765c4a392380d97eb6313af320fc86b"
server_key_version: "encryptionkey@0"
I20260808 00:41:30.766223 4596 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:30.766899 4617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:30.767105 4596 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:30.767198 4596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "0ce38fc80f734d85aa7d8ee645e74ffb"
format_stamp: "Formatted at 2026-08-08 00:41:30 on dist-test-slave-gjw7"
server_key: "2055500807fe991f3be474a479784127"
server_key_iv: "1765c4a392380d97eb6313af320fc86b"
server_key_version: "encryptionkey@0"
I20260808 00:41:30.767263 4596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:30.782388 4596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:30.784543 4596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:30.784682 4596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:30.788836 4596 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43409
I20260808 00:41:30.788875 4700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43409 every 8 connection(s)
I20260808 00:41:30.789216 4596 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:30.789482 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4596
I20260808 00:41:30.789597 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:30.789789 1856 external_mini_cluster.cc:1428] Setting key 0a7f7a222dd4b33511ce5e8e53526b0d
I20260808 00:41:30.790349 4701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:30.793489 4701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb: Bootstrap starting.
I20260808 00:41:30.794190 4701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:30.794485 4701 log.cc:824] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:30.795303 4701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb: No bootstrap required, opened a new log
I20260808 00:41:30.795359 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:30.790313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50315 (local address 127.0.0.1:43409)
0808 00:41:30.790516 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:30.790525 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:30.790551 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:41:30.790996 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:30.791003 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:30.791050 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:30.791165 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:30.791416 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:30.792325 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:30.793406 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:30.793841 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:30.793928 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:30.794619 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:30.794624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:30.794624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:30.794824 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:30.794866 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:30.794883 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:30.794967 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:30.795101 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:41:30.795135 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":71,"threads_started":1}
I20260808 00:41:30.796386 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:30.797874 4701 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } }
I20260808 00:41:30.798048 4701 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:30.798094 4701 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ce38fc80f734d85aa7d8ee645e74ffb, State: Initialized, Role: FOLLOWER
I20260808 00:41:30.798235 4701 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } }
I20260808 00:41:30.798328 4701 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:30.798363 4701 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:30.798413 4701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:30.798949 4701 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } }
I20260808 00:41:30.799060 4701 leader_election.cc:302] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ce38fc80f734d85aa7d8ee645e74ffb; no voters:
I20260808 00:41:30.799242 4701 leader_election.cc:288] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:30.799269 4714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:30.799431 4714 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [term 1 LEADER]: Becoming Leader. State: Replica: 0ce38fc80f734d85aa7d8ee645e74ffb, State: Running, Role: LEADER
I20260808 00:41:30.799497 4701 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:30.799522 4714 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } }
I20260808 00:41:30.799870 4716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ce38fc80f734d85aa7d8ee645e74ffb. Latest consensus state: current_term: 1 leader_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } } }
I20260808 00:41:30.799966 4716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:30.799877 4715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ce38fc80f734d85aa7d8ee645e74ffb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43409 } } }
I20260808 00:41:30.800170 4728 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:30.800211 4715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:30.800700 4728 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:30.801223 4705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:30.798005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38697 (local address 127.0.0.1:43409)
0808 00:41:30.798044 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:30.798045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:30.798168 (+ 123us) server_negotiation.cc:407] Connection header received
0808 00:41:30.798252 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:30.798253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:30.798260 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:30.798295 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:30.798471 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:30.798938 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:30.799904 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:30.800165 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:30.800229 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:30.800972 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:30.800974 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:30.800974 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:30.800978 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:30.800993 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:30.800996 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:30.801043 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:30.801143 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 00:41:30.801154 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:30.802372 4728 catalog_manager.cc:1347] Generated new cluster ID: 1b6b1f44b0404cbfacab3aeedd7bc6d8
I20260808 00:41:30.802433 4728 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:30.813766 4728 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:30.814277 4728 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:30.817376 4728 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ce38fc80f734d85aa7d8ee645e74ffb: Generated new TSK 0
I20260808 00:41:30.817545 4728 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:30.820219 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4596
W20260808 00:41:30.830749 4562 connection.cc:535] client connection to 127.0.0.1:43409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43409): Connection reset by peer (error 104)
2026-08-08T00:41:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:30 dist-test-slave-gjw7 krb5kdc[4754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:30 dist-test-slave-gjw7 krb5kdc[4754](info): set up 2 sockets
Aug 08 00:41:30 dist-test-slave-gjw7 krb5kdc[4754](info): commencing operation
krb5kdc: starting...
W20260808 00:41:31.997529 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:32 dist-test-slave-gjw7 krb5kdc[4754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:32.113495 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:32.193878 4778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:32.194084 4778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:32.194118 4778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:32.195242 4778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:32.195284 4778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:32.195304 4778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:32.195320 4778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:32.195351 4778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:32.196827 4778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42921
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:32.197072 4778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:32.197271 4778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 00:41:32.202066 4778 server_base.cc:1034] running on GCE node
W20260808 00:41:32.202034 4787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:32.202023 4785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:32.202010 4784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:32.202476 4778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:32.202735 4778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:32.203873 4778 hybrid_clock.cc:648] HybridClock initialized: now 1786149692203857 us; error 34 us; skew 500 ppm
Aug 08 00:41:32 dist-test-slave-gjw7 krb5kdc[4754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149692, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:32.206243 4778 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:32.206624 4778 webserver.cc:460] Webserver started at http://127.0.0.1:34647/ using document root <none> and password file <none>
I20260808 00:41:32.206837 4778 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:32.206885 4778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:32.207008 4778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:32.207995 4778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "1b903ba3d48e4b0e85f02006a737431c"
format_stamp: "Formatted at 2026-08-08 00:41:32 on dist-test-slave-gjw7"
server_key: "2eed1dc07b7d35d61913cc8a9377cbaa"
server_key_iv: "fd95d472c98232196ebb1b5f646b9b8a"
server_key_version: "encryptionkey@0"
I20260808 00:41:32.208318 4778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "1b903ba3d48e4b0e85f02006a737431c"
format_stamp: "Formatted at 2026-08-08 00:41:32 on dist-test-slave-gjw7"
server_key: "2eed1dc07b7d35d61913cc8a9377cbaa"
server_key_iv: "fd95d472c98232196ebb1b5f646b9b8a"
server_key_version: "encryptionkey@0"
I20260808 00:41:32.209527 4778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:32.210204 4804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:32.210359 4778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:32.210443 4778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "1b903ba3d48e4b0e85f02006a737431c"
format_stamp: "Formatted at 2026-08-08 00:41:32 on dist-test-slave-gjw7"
server_key: "2eed1dc07b7d35d61913cc8a9377cbaa"
server_key_iv: "fd95d472c98232196ebb1b5f646b9b8a"
server_key_version: "encryptionkey@0"
I20260808 00:41:32.210500 4778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:32.213656 4778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:32.215664 4778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:32.215790 4778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:32.219575 4778 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42921
I20260808 00:41:32.219616 4900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42921 every 8 connection(s)
I20260808 00:41:32.219901 4778 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:32.220889 4902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:32.223067 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c: Bootstrap starting.
I20260808 00:41:32.223603 4902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:32.223837 4902 log.cc:824] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:32.224439 4902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c: No bootstrap required, opened a new log
I20260808 00:41:32.226078 4902 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260808 00:41:32.226214 4902 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:32.226251 4902 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b903ba3d48e4b0e85f02006a737431c, State: Initialized, Role: FOLLOWER
I20260808 00:41:32.226356 4902 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260808 00:41:32.226406 4902 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:32.226425 4902 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:32.226456 4902 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:32.226977 4902 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260808 00:41:32.227083 4902 leader_election.cc:302] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b903ba3d48e4b0e85f02006a737431c; no voters:
I20260808 00:41:32.227260 4902 leader_election.cc:288] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:32.227370 4907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:32.227517 4902 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:32.227918 4907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [term 1 LEADER]: Becoming Leader. State: Replica: 1b903ba3d48e4b0e85f02006a737431c, State: Running, Role: LEADER
I20260808 00:41:32.228065 4907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260808 00:41:32.228096 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4778
I20260808 00:41:32.228215 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:32.228423 1856 external_mini_cluster.cc:1428] Setting key 04c737ea51571ffc3339e6a0b95de180
I20260808 00:41:32.228667 4926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b903ba3d48e4b0e85f02006a737431c. Latest consensus state: current_term: 1 leader_uuid: "1b903ba3d48e4b0e85f02006a737431c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } } }
I20260808 00:41:32.228752 4926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:32.228839 4911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b903ba3d48e4b0e85f02006a737431c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b903ba3d48e4b0e85f02006a737431c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } } }
I20260808 00:41:32.228909 4911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:32.228988 4944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:32.229708 4944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:32.230911 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:32.229090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34275 (local address 127.0.0.1:42921)
0808 00:41:32.229252 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:32.229262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:32.229321 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:41:32.229459 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:32.229466 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:32.229497 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:32.229639 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:32.230567 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:32.230571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:32.230571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:32.230574 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:32.230612 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:32.230624 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:32.230676 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:32.230740 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 00:41:32.230773 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260808 00:41:32.231792 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:32.234274 4946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:32.233020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43605 (local address 127.0.0.1:42921)
0808 00:41:32.233051 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:32.233053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:32.233161 (+ 108us) server_negotiation.cc:407] Connection header received
0808 00:41:32.233233 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:32.233233 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:32.233237 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:32.233280 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:32.233977 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:32.233978 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:32.233978 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:32.233982 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:32.233996 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:32.234000 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:32.234048 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:32.234125 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 00:41:32.234139 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:41:32.234303 4944 catalog_manager.cc:1347] Generated new cluster ID: 928b23a787454802a3c2f8ec5c86ff31
I20260808 00:41:32.234349 4944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:32.241087 4944 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:32.241240 4944 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:32.243772 4944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b903ba3d48e4b0e85f02006a737431c: Generated new TSK 0
I20260808 00:41:32.243932 4944 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:32.246009 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4778
2026-08-08T00:41:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1431 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:32 dist-test-slave-gjw7 krb5kdc[4960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:32 dist-test-slave-gjw7 krb5kdc[4960](info): set up 2 sockets
Aug 08 00:41:32 dist-test-slave-gjw7 krb5kdc[4960](info): commencing operation
krb5kdc: starting...
W20260808 00:41:33.437901 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:33.554298 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40879
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40879
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:33.639632 4986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:33.639850 4986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:33.639873 4986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:33.641052 4986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:33.641103 4986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:33.641119 4986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:33.641134 4986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:33.641162 4986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:33.642735 4986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40879
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:33.642990 4986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:33.643220 4986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:33.648026 4997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:33.648000 4995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:33.647994 4993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:33.648156 4986 server_base.cc:1034] running on GCE node
I20260808 00:41:33.648468 4986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:33.648768 4986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:33.649911 4986 hybrid_clock.cc:648] HybridClock initialized: now 1786149693649900 us; error 33 us; skew 500 ppm
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[4960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:33.652302 4986 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:33.652724 4986 webserver.cc:460] Webserver started at http://127.0.0.1:34059/ using document root <none> and password file <none>
I20260808 00:41:33.652923 4986 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:33.652963 4986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:33.653048 4986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:33.654099 4986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "084e429e09c74e77b90866f6f79086f7"
format_stamp: "Formatted at 2026-08-08 00:41:33 on dist-test-slave-gjw7"
server_key: "5368e6254a1fc2354f7a7d8c863cab7f"
server_key_iv: "49c5742b2d7e2f7ea20d1bb5c4eb8c52"
server_key_version: "encryptionkey@0"
I20260808 00:41:33.654404 4986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "084e429e09c74e77b90866f6f79086f7"
format_stamp: "Formatted at 2026-08-08 00:41:33 on dist-test-slave-gjw7"
server_key: "5368e6254a1fc2354f7a7d8c863cab7f"
server_key_iv: "49c5742b2d7e2f7ea20d1bb5c4eb8c52"
server_key_version: "encryptionkey@0"
I20260808 00:41:33.655575 4986 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:33.656258 5013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:33.656421 4986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:33.656507 4986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "084e429e09c74e77b90866f6f79086f7"
format_stamp: "Formatted at 2026-08-08 00:41:33 on dist-test-slave-gjw7"
server_key: "5368e6254a1fc2354f7a7d8c863cab7f"
server_key_iv: "49c5742b2d7e2f7ea20d1bb5c4eb8c52"
server_key_version: "encryptionkey@0"
I20260808 00:41:33.656564 4986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:33.668790 4986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:33.670714 4986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:33.670833 4986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:33.675181 4986 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40879
I20260808 00:41:33.675244 5093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40879 every 8 connection(s)
I20260808 00:41:33.675576 4986 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:33.676566 5095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:33.678853 5095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: Bootstrap starting.
I20260808 00:41:33.679028 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 4986
I20260808 00:41:33.679136 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:33.679322 1856 external_mini_cluster.cc:1428] Setting key 7942cc0f6035e81f655057a6ac168155
I20260808 00:41:33.679486 5095 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:33.679813 5095 log.cc:824] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:33.680558 5095 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: No bootstrap required, opened a new log
I20260808 00:41:33.682655 5095 raft_consensus.cc:348] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } }
I20260808 00:41:33.682840 5095 raft_consensus.cc:374] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:33.682874 5095 raft_consensus.cc:729] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 084e429e09c74e77b90866f6f79086f7, State: Initialized, Role: FOLLOWER
I20260808 00:41:33.682992 5095 consensus_queue.cc:260] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } }
I20260808 00:41:33.683060 5095 raft_consensus.cc:388] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:33.683085 5095 raft_consensus.cc:482] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:33.683125 5095 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:33.683657 5095 raft_consensus.cc:504] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } }
I20260808 00:41:33.683744 5095 leader_election.cc:302] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 084e429e09c74e77b90866f6f79086f7; no voters:
I20260808 00:41:33.683948 5095 leader_election.cc:288] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:33.683987 5106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:33.684087 5106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [term 1 LEADER]: Becoming Leader. State: Replica: 084e429e09c74e77b90866f6f79086f7, State: Running, Role: LEADER
I20260808 00:41:33.684195 5106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } }
I20260808 00:41:33.684201 5095 sys_catalog.cc:564] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:33.684659 5107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "084e429e09c74e77b90866f6f79086f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } } }
I20260808 00:41:33.684680 5108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 084e429e09c74e77b90866f6f79086f7. Latest consensus state: current_term: 1 leader_uuid: "084e429e09c74e77b90866f6f79086f7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "084e429e09c74e77b90866f6f79086f7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40879 } } }
I20260808 00:41:33.684830 5108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:33.685046 5107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7 [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:33.686228 5126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:33.686285 5126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:33.686332 5115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:33.686491 5115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[4960](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:41:33.687815 5115 catalog_manager.cc:1347] Generated new cluster ID: 7feaa3f4d7a8468cba66f3899a0fbc23
I20260808 00:41:33.687876 5115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:33.691293 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:33.679793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32925 (local address 127.0.0.1:40879)
0808 00:41:33.679964 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:33.679973 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:33.679998 (+ 25us) server_negotiation.cc:407] Connection header received
0808 00:41:33.680443 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:33.680449 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:33.680495 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:33.680618 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:33.681386 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:33.682282 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:33.683305 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:33.683734 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:33.683847 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:33.687192 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:33.687196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:33.687197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:33.687211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:33.689272 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:33.689709 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:33.689712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:33.689712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:33.689815 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:33.690016 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:33.690017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:33.690017 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:33.690298 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:33.690872 (+ 574us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:33.690954 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 00:41:33.691086 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260808 00:41:33.692404 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:33.700848 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:33.694039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45307 (local address 127.0.0.1:40879)
0808 00:41:33.694075 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:33.694076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:33.694208 (+ 132us) server_negotiation.cc:407] Connection header received
0808 00:41:33.694258 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:33.694258 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:33.694270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:33.694326 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:33.694909 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:33.695685 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:33.696672 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:33.696964 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:33.697027 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:33.698301 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:33.698302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:33.698303 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:33.698309 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:33.699770 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:33.700135 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:33.700136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:33.700137 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:33.700197 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:33.700379 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:33.700380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:33.700380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:33.700489 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:33.700575 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:33.700680 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 00:41:33.700758 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:33.707118 5115 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:33.707801 5115 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:33.715497 5115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 084e429e09c74e77b90866f6f79086f7: Generated new TSK 0
I20260808 00:41:33.715673 5115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:33.719808 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 4986
2026-08-08T00:41:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1470 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[5146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[5146](info): set up 2 sockets
Aug 08 00:41:33 dist-test-slave-gjw7 krb5kdc[5146](info): commencing operation
krb5kdc: starting...
W20260808 00:41:34.906605 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:34 dist-test-slave-gjw7 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:35.021638 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:35.103678 5165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:35.103892 5165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:35.103915 5165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:35.105031 5165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:35.105079 5165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:35.105095 5165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:35.105124 5165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:35.105150 5165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:35.106722 5165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:35.106964 5165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:35.107189 5165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:35.111862 5174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:35.111919 5179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:35.112036 5165 server_base.cc:1034] running on GCE node
W20260808 00:41:35.111878 5176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:35.112318 5165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:35.112571 5165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:35.113696 5165 hybrid_clock.cc:648] HybridClock initialized: now 1786149695113691 us; error 39 us; skew 500 ppm
Aug 08 00:41:35 dist-test-slave-gjw7 krb5kdc[5146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149695, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:35.116182 5165 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:35.116585 5165 webserver.cc:460] Webserver started at http://127.0.0.1:45213/ using document root <none> and password file <none>
I20260808 00:41:35.116777 5165 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:35.116819 5165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:35.116955 5165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:35.118024 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "9c4075df571549f9908f63d0d9a8b453"
format_stamp: "Formatted at 2026-08-08 00:41:35 on dist-test-slave-gjw7"
server_key: "056ec1597cb61e4d640eb3460af3f308"
server_key_iv: "42a9ce5d1418730c13995940532ca900"
server_key_version: "encryptionkey@0"
I20260808 00:41:35.118347 5165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "9c4075df571549f9908f63d0d9a8b453"
format_stamp: "Formatted at 2026-08-08 00:41:35 on dist-test-slave-gjw7"
server_key: "056ec1597cb61e4d640eb3460af3f308"
server_key_iv: "42a9ce5d1418730c13995940532ca900"
server_key_version: "encryptionkey@0"
I20260808 00:41:35.119630 5165 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:35.120388 5192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:35.120541 5165 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:35.120627 5165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "9c4075df571549f9908f63d0d9a8b453"
format_stamp: "Formatted at 2026-08-08 00:41:35 on dist-test-slave-gjw7"
server_key: "056ec1597cb61e4d640eb3460af3f308"
server_key_iv: "42a9ce5d1418730c13995940532ca900"
server_key_version: "encryptionkey@0"
I20260808 00:41:35.120688 5165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:35.132086 5165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:35.134101 5165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:35.134233 5165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:35.138096 5165 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39653
I20260808 00:41:35.138136 5271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39653 every 8 connection(s)
I20260808 00:41:35.138438 5165 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:35.139410 5272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:35.141806 5272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453: Bootstrap starting.
I20260808 00:41:35.142411 5272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:35.142683 5272 log.cc:824] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:35.143424 5272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453: No bootstrap required, opened a new log
I20260808 00:41:35.145020 5272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260808 00:41:35.145157 5272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:35.145180 5272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c4075df571549f9908f63d0d9a8b453, State: Initialized, Role: FOLLOWER
I20260808 00:41:35.145354 5272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260808 00:41:35.145432 5272 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:35.145467 5272 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:35.145514 5272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:35.146008 5272 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260808 00:41:35.146130 5272 leader_election.cc:302] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c4075df571549f9908f63d0d9a8b453; no voters:
I20260808 00:41:35.146263 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5165
I20260808 00:41:35.146307 5272 leader_election.cc:288] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:35.146346 5277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:35.146394 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:35.146487 5277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [term 1 LEADER]: Becoming Leader. State: Replica: 9c4075df571549f9908f63d0d9a8b453, State: Running, Role: LEADER
I20260808 00:41:35.146584 1856 external_mini_cluster.cc:1428] Setting key 2f44eb73569c34674e24996c20d9d922
I20260808 00:41:35.146618 5272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:35.146623 5277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260808 00:41:35.147058 5280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c4075df571549f9908f63d0d9a8b453. Latest consensus state: current_term: 1 leader_uuid: "9c4075df571549f9908f63d0d9a8b453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260808 00:41:35.147145 5280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:35.147122 5278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c4075df571549f9908f63d0d9a8b453" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c4075df571549f9908f63d0d9a8b453" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260808 00:41:35.147194 5278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:35.147393 5291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:35.147905 5291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:35.149576 5291 catalog_manager.cc:1347] Generated new cluster ID: 1566d37243234f60832d96a2c26259ba
I20260808 00:41:35.149667 5291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:35.153108 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:35.147060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48699 (local address 127.0.0.1:39653)
0808 00:41:35.148173 (+ 1113us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:35.148183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:35.148442 (+ 259us) server_negotiation.cc:407] Connection header received
0808 00:41:35.148598 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:35.148605 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:35.148666 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:35.148818 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:35.149035 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:35.150446 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:35.151338 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:35.151771 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:35.151870 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:35.152501 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:35.152505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:35.152505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:35.152671 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:35.152702 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:35.152714 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:35.152771 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:35.152847 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 00:41:35.152910 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":1064,"thread_start_us":85,"threads_started":1}
I20260808 00:41:35.154024 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:35.159902 5288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:35.155383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45659 (local address 127.0.0.1:39653)
0808 00:41:35.155417 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:35.155418 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:35.157459 (+ 2041us) server_negotiation.cc:407] Connection header received
0808 00:41:35.157513 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:35.157514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:35.157524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:35.157568 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:35.157683 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:35.158148 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:35.158859 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:35.159052 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:35.159108 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:35.159661 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:35.159662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:35.159662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:35.159680 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:35.159694 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:35.159698 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:35.159746 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:35.159806 (+ 60us) server_negotiation.cc:299] Negotiation successful
0808 00:41:35.159828 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:35.165436 5291 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:35.166096 5291 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:35.178943 5291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c4075df571549f9908f63d0d9a8b453: Generated new TSK 0
I20260808 00:41:35.179133 5291 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:35.184770 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5165
2026-08-08T00:41:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1466 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:35 dist-test-slave-gjw7 krb5kdc[5324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:35 dist-test-slave-gjw7 krb5kdc[5324](info): set up 2 sockets
Aug 08 00:41:35 dist-test-slave-gjw7 krb5kdc[5324](info): commencing operation
krb5kdc: starting...
W20260808 00:41:36.374624 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:36 dist-test-slave-gjw7 krb5kdc[5324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:36.493574 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:36.577798 5351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:36.578008 5351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:36.578039 5351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:36.579155 5351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:36.579197 5351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:36.579216 5351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:36.579232 5351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:36.579249 5351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:36.580721 5351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37815
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:36.580964 5351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:36.581162 5351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:36.585942 5360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:36.585915 5356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:36.585983 5362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:36.585927 5351 server_base.cc:1034] running on GCE node
I20260808 00:41:36.586419 5351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:36.586673 5351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:36.587832 5351 hybrid_clock.cc:648] HybridClock initialized: now 1786149696587813 us; error 34 us; skew 500 ppm
Aug 08 00:41:36 dist-test-slave-gjw7 krb5kdc[5324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149696, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:36.594328 5351 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:36.594761 5351 webserver.cc:460] Webserver started at http://127.0.0.1:42097/ using document root <none> and password file <none>
I20260808 00:41:36.594959 5351 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:36.595007 5351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:36.595134 5351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:36.596196 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "a16b3090f0184ac497172da41cda6d00"
format_stamp: "Formatted at 2026-08-08 00:41:36 on dist-test-slave-gjw7"
server_key: "502c1efbcb50aeefeefcefb5a823e4c9"
server_key_iv: "60c3c9818accbb7c3d3e19478ced4c60"
server_key_version: "encryptionkey@0"
I20260808 00:41:36.596540 5351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "a16b3090f0184ac497172da41cda6d00"
format_stamp: "Formatted at 2026-08-08 00:41:36 on dist-test-slave-gjw7"
server_key: "502c1efbcb50aeefeefcefb5a823e4c9"
server_key_iv: "60c3c9818accbb7c3d3e19478ced4c60"
server_key_version: "encryptionkey@0"
I20260808 00:41:36.597834 5351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:36.598598 5375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:36.598740 5351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:36.598830 5351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "a16b3090f0184ac497172da41cda6d00"
format_stamp: "Formatted at 2026-08-08 00:41:36 on dist-test-slave-gjw7"
server_key: "502c1efbcb50aeefeefcefb5a823e4c9"
server_key_iv: "60c3c9818accbb7c3d3e19478ced4c60"
server_key_version: "encryptionkey@0"
I20260808 00:41:36.598894 5351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:36.602303 5351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:36.604038 5351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:36.604171 5351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:36.610286 5351 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37815
I20260808 00:41:36.610340 5464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37815 every 8 connection(s)
I20260808 00:41:36.610646 5351 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:36.611584 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:36.611817 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5351
I20260808 00:41:36.611927 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:36.612094 1856 external_mini_cluster.cc:1428] Setting key 7a0634d1e17a84c5c4d6c59f8209cee3
I20260808 00:41:36.614578 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:36.612613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40781 (local address 127.0.0.1:37815)
0808 00:41:36.612812 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:36.612832 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:36.612867 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:41:36.613425 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:36.613434 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:36.613470 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:36.613617 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:36.614186 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:36.614189 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:36.614189 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:36.614194 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:36.614239 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:36.614253 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:36.614326 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:36.614366 (+ 40us) server_negotiation.cc:299] Negotiation successful
0808 00:41:36.614406 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260808 00:41:36.614634 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00: Bootstrap starting.
I20260808 00:41:36.615302 5465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:36.615595 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:36.615612 5465 log.cc:824] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:36.616251 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00: No bootstrap required, opened a new log
I20260808 00:41:36.617930 5469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:36.616939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50053 (local address 127.0.0.1:37815)
0808 00:41:36.616994 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:36.616996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:36.617139 (+ 143us) server_negotiation.cc:407] Connection header received
0808 00:41:36.617199 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:36.617200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:36.617206 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:36.617247 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:36.617748 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:36.617749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:36.617749 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:36.617752 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:36.617763 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:36.617767 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:36.617807 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:36.617881 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 00:41:36.617890 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:41:36.618494 5465 raft_consensus.cc:348] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } }
I20260808 00:41:36.618630 5465 raft_consensus.cc:374] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:36.618650 5465 raft_consensus.cc:729] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a16b3090f0184ac497172da41cda6d00, State: Initialized, Role: FOLLOWER
I20260808 00:41:36.618762 5465 consensus_queue.cc:260] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } }
I20260808 00:41:36.618839 5465 raft_consensus.cc:388] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:36.618870 5465 raft_consensus.cc:482] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:36.618916 5465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:36.619415 5465 raft_consensus.cc:504] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } }
I20260808 00:41:36.619504 5465 leader_election.cc:302] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a16b3090f0184ac497172da41cda6d00; no voters:
I20260808 00:41:36.619665 5465 leader_election.cc:288] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:36.619782 5478 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:36.619894 5465 sys_catalog.cc:564] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:36.619935 5478 raft_consensus.cc:686] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [term 1 LEADER]: Becoming Leader. State: Replica: a16b3090f0184ac497172da41cda6d00, State: Running, Role: LEADER
I20260808 00:41:36.620069 5478 consensus_queue.cc:237] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } }
I20260808 00:41:36.620434 5479 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a16b3090f0184ac497172da41cda6d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } } }
I20260808 00:41:36.620431 5481 sys_catalog.cc:455] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a16b3090f0184ac497172da41cda6d00. Latest consensus state: current_term: 1 leader_uuid: "a16b3090f0184ac497172da41cda6d00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a16b3090f0184ac497172da41cda6d00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37815 } } }
I20260808 00:41:36.620551 5479 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:36.620559 5481 sys_catalog.cc:458] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:36.620774 5492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:36.621260 5492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:36.622771 5492 catalog_manager.cc:1347] Generated new cluster ID: df9a9d11314443f8b6e4506a625060ca
I20260808 00:41:36.622814 5492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:36.627681 5492 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:36.627815 5492 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:36.632795 5492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a16b3090f0184ac497172da41cda6d00: Generated new TSK 0
I20260808 00:41:36.632961 5492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:36.639909 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5351
2026-08-08T00:41:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:36 dist-test-slave-gjw7 krb5kdc[5510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:36 dist-test-slave-gjw7 krb5kdc[5510](info): set up 2 sockets
Aug 08 00:41:36 dist-test-slave-gjw7 krb5kdc[5510](info): commencing operation
krb5kdc: starting...
W20260808 00:41:37.813210 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:37 dist-test-slave-gjw7 krb5kdc[5510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:37.930147 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:38.012362 5537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:38.012575 5537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:38.012606 5537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:38.013772 5537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:38.013811 5537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:38.013831 5537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:38.013846 5537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:38.013863 5537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:38.015344 5537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:38.015587 5537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:38.015796 5537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:38.020440 5544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:38.020493 5548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:38.020448 5543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:38.020711 5537 server_base.cc:1034] running on GCE node
I20260808 00:41:38.020965 5537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:38.021230 5537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:38.022372 5537 hybrid_clock.cc:648] HybridClock initialized: now 1786149698022359 us; error 31 us; skew 500 ppm
Aug 08 00:41:38 dist-test-slave-gjw7 krb5kdc[5510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149698, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:38.024735 5537 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:38.025156 5537 webserver.cc:460] Webserver started at http://127.0.0.1:42373/ using document root <none> and password file <none>
I20260808 00:41:38.025398 5537 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:38.025444 5537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:38.025565 5537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:38.026610 5537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "c6bc028d503a44ef8bb7ce75768245ea"
format_stamp: "Formatted at 2026-08-08 00:41:38 on dist-test-slave-gjw7"
server_key: "875424ecbd6be33489950ff37898f9a2"
server_key_iv: "05e225f61ca0dd52c09ed0de2ebe9035"
server_key_version: "encryptionkey@0"
I20260808 00:41:38.026955 5537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "c6bc028d503a44ef8bb7ce75768245ea"
format_stamp: "Formatted at 2026-08-08 00:41:38 on dist-test-slave-gjw7"
server_key: "875424ecbd6be33489950ff37898f9a2"
server_key_iv: "05e225f61ca0dd52c09ed0de2ebe9035"
server_key_version: "encryptionkey@0"
I20260808 00:41:38.028156 5537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:38.028812 5561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:38.028963 5537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:38.029044 5537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "c6bc028d503a44ef8bb7ce75768245ea"
format_stamp: "Formatted at 2026-08-08 00:41:38 on dist-test-slave-gjw7"
server_key: "875424ecbd6be33489950ff37898f9a2"
server_key_iv: "05e225f61ca0dd52c09ed0de2ebe9035"
server_key_version: "encryptionkey@0"
I20260808 00:41:38.029112 5537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:38.040838 5537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:38.042641 5537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:38.042766 5537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:38.046854 5537 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33231
I20260808 00:41:38.046887 5645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33231 every 8 connection(s)
I20260808 00:41:38.047217 5537 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:38.048161 5646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:38.050546 5646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea: Bootstrap starting.
I20260808 00:41:38.051112 5646 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:38.051383 5646 log.cc:824] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:38.052044 5646 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea: No bootstrap required, opened a new log
I20260808 00:41:38.053675 5646 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260808 00:41:38.053817 5646 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:38.053862 5646 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6bc028d503a44ef8bb7ce75768245ea, State: Initialized, Role: FOLLOWER
I20260808 00:41:38.054005 5646 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260808 00:41:38.054085 5646 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:38.054111 5646 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:38.054159 5646 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:38.054661 5646 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260808 00:41:38.054750 5646 leader_election.cc:302] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6bc028d503a44ef8bb7ce75768245ea; no voters:
I20260808 00:41:38.054911 5646 leader_election.cc:288] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:38.054950 5649 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:38.054961 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5537
I20260808 00:41:38.055065 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:38.055066 5649 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [term 1 LEADER]: Becoming Leader. State: Replica: c6bc028d503a44ef8bb7ce75768245ea, State: Running, Role: LEADER
I20260808 00:41:38.055138 5649 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } }
I20260808 00:41:38.055168 5646 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:38.055295 1856 external_mini_cluster.cc:1428] Setting key ad7e0ec69741c91ea3bf25d952b2d388
I20260808 00:41:38.055631 5649 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6bc028d503a44ef8bb7ce75768245ea. Latest consensus state: current_term: 1 leader_uuid: "c6bc028d503a44ef8bb7ce75768245ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } } }
I20260808 00:41:38.055773 5649 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:38.055737 5652 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6bc028d503a44ef8bb7ce75768245ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6bc028d503a44ef8bb7ce75768245ea" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33231 } } }
I20260808 00:41:38.055889 5652 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:38.056056 5667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:38.056648 5667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:38.058182 5667 catalog_manager.cc:1347] Generated new cluster ID: 78cadecc33114f9fabf51057c6102d80
I20260808 00:41:38.058238 5667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:38.064193 5667 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:38.064662 5667 catalog_manager.cc:1504] Loading token signing keys...
Aug 08 00:41:38 dist-test-slave-gjw7 krb5kdc[5510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:41:38.069626 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:38.056035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:38757 (local address 127.0.0.1:33231)
0808 00:41:38.056462 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:38.056471 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:38.056504 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:41:38.056623 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:38.056642 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:38.056708 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:38.056842 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:38.057682 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:38.061990 (+ 4308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:38.062712 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:38.063175 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:38.063323 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:38.066097 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:38.066101 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:38.066101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:38.066114 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:38.067991 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:38.068413 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:38.068417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:38.068417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:38.068525 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:38.068721 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:38.068722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:38.068722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:38.069014 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:38.069171 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:38.069252 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 00:41:38.069391 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":91,"threads_started":1}
I20260808 00:41:38.070547 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:38.072548 5667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6bc028d503a44ef8bb7ce75768245ea: Generated new TSK 0
I20260808 00:41:38.072726 5667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:38.079250 5670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:38.072488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:49009 (local address 127.0.0.1:33231)
0808 00:41:38.072538 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:38.072540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:38.072819 (+ 279us) server_negotiation.cc:407] Connection header received
0808 00:41:38.072900 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:38.072901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:38.072912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:38.072974 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:38.073578 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:38.074400 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:38.075109 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:38.075441 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:38.075506 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:38.076711 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:38.076713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:38.076713 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:38.076719 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:38.078176 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:38.078541 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:38.078543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:38.078543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:38.078605 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:38.078777 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:38.078777 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:38.078778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:38.078898 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:38.079000 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:38.079084 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 00:41:38.079153 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:41:38.081389 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5537
W20260808 00:41:38.093194 5508 connection.cc:535] client connection to 127.0.0.1:33231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33231): Connection reset by peer (error 104)
2026-08-08T00:41:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:38 dist-test-slave-gjw7 krb5kdc[5696](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:38 dist-test-slave-gjw7 krb5kdc[5696](info): set up 2 sockets
Aug 08 00:41:38 dist-test-slave-gjw7 krb5kdc[5696](info): commencing operation
krb5kdc: starting...
W20260808 00:41:39.257468 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:39 dist-test-slave-gjw7 krb5kdc[5696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:39.381284 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:39.461804 5719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:39.462002 5719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:39.462033 5719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:39.463145 5719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:39.463186 5719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:39.463203 5719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:39.463219 5719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:39.463234 5719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:39.464656 5719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:39.464910 5719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:39.465111 5719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:39.469592 5725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:39.469604 5736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:39.469805 5719 server_base.cc:1034] running on GCE node
W20260808 00:41:39.469642 5729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:39.470086 5719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:39.470338 5719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:39.471475 5719 hybrid_clock.cc:648] HybridClock initialized: now 1786149699471452 us; error 39 us; skew 500 ppm
Aug 08 00:41:39 dist-test-slave-gjw7 krb5kdc[5696](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149699, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:39.473893 5719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:39.474309 5719 webserver.cc:460] Webserver started at http://127.0.0.1:36679/ using document root <none> and password file <none>
I20260808 00:41:39.474517 5719 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:39.474566 5719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:39.474689 5719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:39.475662 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "680b34639e324ceeb6842959fa64cb4e"
format_stamp: "Formatted at 2026-08-08 00:41:39 on dist-test-slave-gjw7"
server_key: "e61872cbd95f804705e2aa6cdbc9a1b3"
server_key_iv: "bb266bf912fd31774e7d59edd8a1f41b"
server_key_version: "encryptionkey@0"
I20260808 00:41:39.475997 5719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "680b34639e324ceeb6842959fa64cb4e"
format_stamp: "Formatted at 2026-08-08 00:41:39 on dist-test-slave-gjw7"
server_key: "e61872cbd95f804705e2aa6cdbc9a1b3"
server_key_iv: "bb266bf912fd31774e7d59edd8a1f41b"
server_key_version: "encryptionkey@0"
I20260808 00:41:39.477218 5719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:41:39.477942 5750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:39.478087 5719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:39.478183 5719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "680b34639e324ceeb6842959fa64cb4e"
format_stamp: "Formatted at 2026-08-08 00:41:39 on dist-test-slave-gjw7"
server_key: "e61872cbd95f804705e2aa6cdbc9a1b3"
server_key_iv: "bb266bf912fd31774e7d59edd8a1f41b"
server_key_version: "encryptionkey@0"
I20260808 00:41:39.478245 5719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:39.490869 5719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:39.492648 5719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:39.492770 5719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:39.496744 5719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45165
I20260808 00:41:39.496779 5840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45165 every 8 connection(s)
I20260808 00:41:39.497102 5719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:39.498042 5841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:39.500305 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e: Bootstrap starting.
I20260808 00:41:39.500946 5841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:39.501220 5841 log.cc:824] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:39.501878 5841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e: No bootstrap required, opened a new log
I20260808 00:41:39.503443 5841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260808 00:41:39.503618 5841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:39.503675 5841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 680b34639e324ceeb6842959fa64cb4e, State: Initialized, Role: FOLLOWER
I20260808 00:41:39.503844 5841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260808 00:41:39.503940 5841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:39.503979 5841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:39.504029 5841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:39.504539 5841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260808 00:41:39.504634 5841 leader_election.cc:302] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 680b34639e324ceeb6842959fa64cb4e; no voters:
I20260808 00:41:39.504796 5841 leader_election.cc:288] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:39.504835 5845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:39.504992 5845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [term 1 LEADER]: Becoming Leader. State: Replica: 680b34639e324ceeb6842959fa64cb4e, State: Running, Role: LEADER
I20260808 00:41:39.505035 5841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:39.505081 5845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } }
I20260808 00:41:39.505666 5850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 680b34639e324ceeb6842959fa64cb4e. Latest consensus state: current_term: 1 leader_uuid: "680b34639e324ceeb6842959fa64cb4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } } }
I20260808 00:41:39.505757 5850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:39.506006 5846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "680b34639e324ceeb6842959fa64cb4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "680b34639e324ceeb6842959fa64cb4e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45165 } } }
I20260808 00:41:39.506059 5859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:39.506261 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5719
I20260808 00:41:39.506146 5846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:39.506398 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:39.506582 1856 external_mini_cluster.cc:1428] Setting key cc3258e1f375aa6d2fc88046f1e38b99
I20260808 00:41:39.506693 5859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:39.508186 5859 catalog_manager.cc:1347] Generated new cluster ID: ba66fdd264e8494db1114dd8edd20386
I20260808 00:41:39.508237 5859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:39.523029 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:39.507042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:42817 (local address 127.0.0.1:45165)
0808 00:41:39.507217 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:39.507226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:39.517540 (+ 10314us) server_negotiation.cc:407] Connection header received
0808 00:41:39.517681 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:39.517690 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:39.517754 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:39.517896 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:39.518141 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:39.519606 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:39.520660 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:39.521320 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:39.521431 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:39.522163 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:39.522167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:39.522168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:39.522411 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:39.522453 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:39.522468 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:39.522559 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:39.522745 (+ 186us) server_negotiation.cc:299] Negotiation successful
0808 00:41:39.522778 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":75,"threads_started":1}
I20260808 00:41:39.523274 5859 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:39.523926 5859 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:39.524093 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:39.529012 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:39.525576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:42501 (local address 127.0.0.1:45165)
0808 00:41:39.525610 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:39.525612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:39.525683 (+ 71us) server_negotiation.cc:407] Connection header received
0808 00:41:39.525737 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:39.525738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:39.525746 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:39.525792 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:39.525940 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:39.526675 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:39.527738 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:39.528053 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:39.528103 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:39.528773 (+ 670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:39.528774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:39.528774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:39.528778 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:39.528793 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:39.528797 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:39.528850 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:39.528927 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 00:41:39.528940 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:39.532065 5859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 680b34639e324ceeb6842959fa64cb4e: Generated new TSK 0
I20260808 00:41:39.532253 5859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:39.538944 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5719
W20260808 00:41:39.553536 5694 connection.cc:535] client connection to 127.0.0.1:45165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45165): Connection reset by peer (error 104)
2026-08-08T00:41:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:39 dist-test-slave-gjw7 krb5kdc[5885](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:39 dist-test-slave-gjw7 krb5kdc[5885](info): set up 2 sockets
Aug 08 00:41:39 dist-test-slave-gjw7 krb5kdc[5885](info): commencing operation
krb5kdc: starting...
W20260808 00:41:40.735942 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:40 dist-test-slave-gjw7 krb5kdc[5885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:40.858089 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46665
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46665
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:40.940891 5898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:40.941097 5898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:40.941118 5898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:40.942267 5898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:40.942312 5898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:40.942324 5898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:40.942335 5898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:40.942345 5898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:40.943771 5898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:40.943969 5898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:40.944145 5898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:40.948894 5910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:40.948901 5908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:40.948943 5907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:40.949155 5898 server_base.cc:1034] running on GCE node
I20260808 00:41:40.949487 5898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:40.949770 5898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:40.950918 5898 hybrid_clock.cc:648] HybridClock initialized: now 1786149700950897 us; error 36 us; skew 500 ppm
Aug 08 00:41:40 dist-test-slave-gjw7 krb5kdc[5885](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149700, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:40.953488 5898 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:40.953922 5898 webserver.cc:460] Webserver started at http://127.0.0.1:39573/ using document root <none> and password file <none>
I20260808 00:41:40.954119 5898 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:40.954174 5898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:40.954303 5898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:40.955358 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "7c96e5aac9e04d0eaccdade4d0771163"
format_stamp: "Formatted at 2026-08-08 00:41:40 on dist-test-slave-gjw7"
server_key: "11005cb9c2dc2f1e62a2b58e8b65f02c"
server_key_iv: "574e68199f5ebe2f9bee72879028b5c8"
server_key_version: "encryptionkey@0"
I20260808 00:41:40.955699 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "7c96e5aac9e04d0eaccdade4d0771163"
format_stamp: "Formatted at 2026-08-08 00:41:40 on dist-test-slave-gjw7"
server_key: "11005cb9c2dc2f1e62a2b58e8b65f02c"
server_key_iv: "574e68199f5ebe2f9bee72879028b5c8"
server_key_version: "encryptionkey@0"
I20260808 00:41:40.956982 5898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:40.957679 5926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:40.957813 5898 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:40.957906 5898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "7c96e5aac9e04d0eaccdade4d0771163"
format_stamp: "Formatted at 2026-08-08 00:41:40 on dist-test-slave-gjw7"
server_key: "11005cb9c2dc2f1e62a2b58e8b65f02c"
server_key_iv: "574e68199f5ebe2f9bee72879028b5c8"
server_key_version: "encryptionkey@0"
I20260808 00:41:40.957974 5898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:40.961193 5898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:40.963035 5898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:40.963160 5898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:40.967090 5898 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46665
I20260808 00:41:40.967134 6013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46665 every 8 connection(s)
I20260808 00:41:40.967444 5898 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:40.968417 6014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:40.970708 6014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163: Bootstrap starting.
I20260808 00:41:40.971232 6014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:40.971472 6014 log.cc:824] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:40.972065 6014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163: No bootstrap required, opened a new log
I20260808 00:41:40.973084 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 5898
I20260808 00:41:40.973198 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:40.973395 1856 external_mini_cluster.cc:1428] Setting key 3b2a7693e8f6053448889fa4a14fda06
I20260808 00:41:40.973855 6014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } }
I20260808 00:41:40.974036 6014 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:40.974081 6014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c96e5aac9e04d0eaccdade4d0771163, State: Initialized, Role: FOLLOWER
I20260808 00:41:40.974210 6014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } }
I20260808 00:41:40.974287 6014 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:40.974315 6014 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:40.974360 6014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:40.974982 6014 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } }
I20260808 00:41:40.975082 6014 leader_election.cc:302] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c96e5aac9e04d0eaccdade4d0771163; no voters:
I20260808 00:41:40.975252 6014 leader_election.cc:288] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:40.975274 6021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:40.975445 6021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [term 1 LEADER]: Becoming Leader. State: Replica: 7c96e5aac9e04d0eaccdade4d0771163, State: Running, Role: LEADER
I20260808 00:41:40.975494 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:40.973882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:33393 (local address 127.0.0.1:46665)
0808 00:41:40.974054 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:40.974064 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:40.974097 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:41:40.974214 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:40.974220 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:40.974250 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:40.974388 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:40.975042 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:40.975046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:40.975046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:40.975050 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:40.975091 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:40.975106 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:40.975161 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:40.975223 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:41:40.975268 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":76,"threads_started":1}
I20260808 00:41:40.975544 6014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:40.975675 6021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } }
I20260808 00:41:40.976281 6023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7c96e5aac9e04d0eaccdade4d0771163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } } }
I20260808 00:41:40.976373 6023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:40.976501 6023 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7c96e5aac9e04d0eaccdade4d0771163. Latest consensus state: current_term: 1 leader_uuid: "7c96e5aac9e04d0eaccdade4d0771163" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c96e5aac9e04d0eaccdade4d0771163" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46665 } } }
I20260808 00:41:40.976557 6023 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:40.976580 6037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:40.976847 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:40.977073 6037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:40.978713 6037 catalog_manager.cc:1347] Generated new cluster ID: 54d17582590842db8202d790a4c5c70f
I20260808 00:41:40.978767 6037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:40.979429 6020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:40.978365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:49319 (local address 127.0.0.1:46665)
0808 00:41:40.978397 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:40.978399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:40.978535 (+ 136us) server_negotiation.cc:407] Connection header received
0808 00:41:40.978583 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:40.978584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:40.978589 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:40.978642 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:40.979220 (+ 578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:40.979221 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:40.979222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:40.979226 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:40.979243 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:40.979248 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:40.979308 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:40.979356 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 00:41:40.979369 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:40.990433 6037 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:40.990593 6037 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:40.993782 6037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7c96e5aac9e04d0eaccdade4d0771163: Generated new TSK 0
I20260808 00:41:40.993937 6037 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:40.998699 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 5898
2026-08-08T00:41:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:41 dist-test-slave-gjw7 krb5kdc[6063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:41 dist-test-slave-gjw7 krb5kdc[6063](info): set up 2 sockets
Aug 08 00:41:41 dist-test-slave-gjw7 krb5kdc[6063](info): commencing operation
krb5kdc: starting...
W20260808 00:41:42.188472 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:42.307545 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34431
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:42.391139 6084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:42.391351 6084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:42.391374 6084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:42.392537 6084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:42.392589 6084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:42.392604 6084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:42.392619 6084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:42.392648 6084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:42.394169 6084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34431
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:42.394411 6084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:42.394626 6084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:42.399384 6094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:42.399498 6096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:42.399488 6084 server_base.cc:1034] running on GCE node
W20260808 00:41:42.399384 6099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:42.399876 6084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:42.400159 6084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:42.401330 6084 hybrid_clock.cc:648] HybridClock initialized: now 1786149702401308 us; error 36 us; skew 500 ppm
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149702, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:42.407413 6084 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:42.407832 6084 webserver.cc:460] Webserver started at http://127.0.0.1:44011/ using document root <none> and password file <none>
I20260808 00:41:42.408028 6084 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:42.408075 6084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:42.408201 6084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:42.409348 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "b0ebee38eabd4bafaff3cdae781f4806"
format_stamp: "Formatted at 2026-08-08 00:41:42 on dist-test-slave-gjw7"
server_key: "6bc789e88f3a9a7bc532fd1281225b55"
server_key_iv: "f6797d8def4fe7b2f2040ff5f9d7f5ec"
server_key_version: "encryptionkey@0"
I20260808 00:41:42.409720 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "b0ebee38eabd4bafaff3cdae781f4806"
format_stamp: "Formatted at 2026-08-08 00:41:42 on dist-test-slave-gjw7"
server_key: "6bc789e88f3a9a7bc532fd1281225b55"
server_key_iv: "f6797d8def4fe7b2f2040ff5f9d7f5ec"
server_key_version: "encryptionkey@0"
I20260808 00:41:42.410984 6084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:41:42.411710 6112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:42.411877 6084 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:42.411964 6084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "b0ebee38eabd4bafaff3cdae781f4806"
format_stamp: "Formatted at 2026-08-08 00:41:42 on dist-test-slave-gjw7"
server_key: "6bc789e88f3a9a7bc532fd1281225b55"
server_key_iv: "f6797d8def4fe7b2f2040ff5f9d7f5ec"
server_key_version: "encryptionkey@0"
I20260808 00:41:42.412030 6084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:42.434681 6084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:42.436815 6084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:42.436956 6084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:42.440920 6084 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34431
I20260808 00:41:42.440960 6194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34431 every 8 connection(s)
I20260808 00:41:42.441306 6084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:42.442278 6195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:42.442463 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6084
I20260808 00:41:42.442586 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:42.442801 1856 external_mini_cluster.cc:1428] Setting key 41eda3c2a510b051ef18d738ab08717f
I20260808 00:41:42.445794 6195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806: Bootstrap starting.
I20260808 00:41:42.446506 6195 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:42.446801 6195 log.cc:824] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:42.447572 6195 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806: No bootstrap required, opened a new log
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 00:41:42.449935 6195 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } }
I20260808 00:41:42.450074 6195 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:42.450107 6195 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ebee38eabd4bafaff3cdae781f4806, State: Initialized, Role: FOLLOWER
I20260808 00:41:42.450279 6195 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } }
I20260808 00:41:42.450352 6195 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:42.450377 6195 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:42.450430 6195 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:42.451007 6195 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } }
I20260808 00:41:42.451121 6195 leader_election.cc:302] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0ebee38eabd4bafaff3cdae781f4806; no voters:
I20260808 00:41:42.451287 6195 leader_election.cc:288] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:42.451330 6203 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:42.451478 6203 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [term 1 LEADER]: Becoming Leader. State: Replica: b0ebee38eabd4bafaff3cdae781f4806, State: Running, Role: LEADER
I20260808 00:41:42.451547 6195 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:42.451629 6203 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } }
I20260808 00:41:42.452093 6208 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0ebee38eabd4bafaff3cdae781f4806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } } }
I20260808 00:41:42.452240 6208 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:42.452467 6203 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0ebee38eabd4bafaff3cdae781f4806. Latest consensus state: current_term: 1 leader_uuid: "b0ebee38eabd4bafaff3cdae781f4806" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0ebee38eabd4bafaff3cdae781f4806" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34431 } } }
I20260808 00:41:42.452589 6203 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:42.452596 6218 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:42.453688 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:42.443310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:50897 (local address 127.0.0.1:34431)
0808 00:41:42.443571 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:42.443580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:42.443609 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:41:42.444069 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:42.444077 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:42.444126 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:42.444240 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:42.445024 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:42.445985 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:42.447001 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:42.447508 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:42.447586 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:42.450079 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:42.450084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:42.450084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:42.450097 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:42.451978 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:42.452403 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:42.452406 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:42.452406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:42.452515 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:42.452781 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:42.452782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:42.452782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:42.453070 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:42.453246 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:42.453393 (+ 147us) server_negotiation.cc:299] Negotiation successful
0808 00:41:42.453487 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":159,"threads_started":1}
I20260808 00:41:42.453831 6218 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:42.454710 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:42.455279 6218 catalog_manager.cc:1347] Generated new cluster ID: 48d148602ce44a78bd7bbd0ba9e4c964
I20260808 00:41:42.455320 6218 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:42.467303 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:42.461657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:60815 (local address 127.0.0.1:34431)
0808 00:41:42.461696 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:42.461699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:42.461868 (+ 169us) server_negotiation.cc:407] Connection header received
0808 00:41:42.461921 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:42.461922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:42.461932 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:42.461976 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:42.462462 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:42.462927 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:42.463601 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:42.463829 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:42.463883 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:42.464953 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:42.464954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:42.464954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:42.464962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:42.466241 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:42.466579 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:42.466580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:42.466580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:42.466629 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:42.466820 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:42.466821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:42.466821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:42.466929 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:42.467017 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:42.467149 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 00:41:42.467218 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:41:42.472365 6218 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:42.473049 6218 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:42.486570 6218 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0ebee38eabd4bafaff3cdae781f4806: Generated new TSK 0
I20260808 00:41:42.486778 6218 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:42.492637 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6084
W20260808 00:41:42.504606 6061 connection.cc:535] client connection to 127.0.0.1:34431 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34431): Connection reset by peer (error 104)
2026-08-08T00:41:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1492 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6248](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6248](info): set up 2 sockets
Aug 08 00:41:42 dist-test-slave-gjw7 krb5kdc[6248](info): commencing operation
krb5kdc: starting...
W20260808 00:41:43.666522 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:43 dist-test-slave-gjw7 krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:43.783274 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:43.865765 6266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:43.865953 6266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:43.865974 6266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:43.867101 6266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:43.867141 6266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:43.867154 6266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:43.867166 6266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:43.867188 6266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:43.868698 6266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:43.868925 6266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:43.869122 6266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:43.873661 6276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:43.873772 6266 server_base.cc:1034] running on GCE node
W20260808 00:41:43.873826 6273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:43.873672 6279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:43.874154 6266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:43.874433 6266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:43.875572 6266 hybrid_clock.cc:648] HybridClock initialized: now 1786149703875559 us; error 31 us; skew 500 ppm
Aug 08 00:41:43 dist-test-slave-gjw7 krb5kdc[6248](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149703, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:43.877959 6266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:43.878347 6266 webserver.cc:460] Webserver started at http://127.0.0.1:36635/ using document root <none> and password file <none>
I20260808 00:41:43.878566 6266 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:43.878623 6266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:43.878756 6266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:43.879735 6266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "4a70724bcb0d4d1fb6d7c243c5009a84"
format_stamp: "Formatted at 2026-08-08 00:41:43 on dist-test-slave-gjw7"
server_key: "9965e0d6374fe83dd84bd37c5b628d20"
server_key_iv: "58f0556d30f4f5ce7385baf0b16775d1"
server_key_version: "encryptionkey@0"
I20260808 00:41:43.880062 6266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "4a70724bcb0d4d1fb6d7c243c5009a84"
format_stamp: "Formatted at 2026-08-08 00:41:43 on dist-test-slave-gjw7"
server_key: "9965e0d6374fe83dd84bd37c5b628d20"
server_key_iv: "58f0556d30f4f5ce7385baf0b16775d1"
server_key_version: "encryptionkey@0"
I20260808 00:41:43.881458 6266 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:43.882334 6290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:43.882568 6266 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 00:41:43.882664 6266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "4a70724bcb0d4d1fb6d7c243c5009a84"
format_stamp: "Formatted at 2026-08-08 00:41:43 on dist-test-slave-gjw7"
server_key: "9965e0d6374fe83dd84bd37c5b628d20"
server_key_iv: "58f0556d30f4f5ce7385baf0b16775d1"
server_key_version: "encryptionkey@0"
I20260808 00:41:43.882737 6266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:43.898278 6266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:43.900036 6266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:43.900146 6266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:43.904029 6266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43917
I20260808 00:41:43.904060 6356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43917 every 8 connection(s)
I20260808 00:41:43.904357 6266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:43.905159 6357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:43.907445 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: Bootstrap starting.
I20260808 00:41:43.908007 6357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:43.908298 6357 log.cc:824] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:43.908288 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6266
I20260808 00:41:43.908401 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:43.908577 1856 external_mini_cluster.cc:1428] Setting key b34fcafc1d65c217f261f9567148a70a
I20260808 00:41:43.908963 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: No bootstrap required, opened a new log
I20260808 00:41:43.910779 6357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } }
I20260808 00:41:43.910929 6357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:43.910951 6357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a70724bcb0d4d1fb6d7c243c5009a84, State: Initialized, Role: FOLLOWER
I20260808 00:41:43.911062 6357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } }
I20260808 00:41:43.911134 6357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:43.911156 6357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:43.911202 6357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:43.911724 6357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } }
I20260808 00:41:43.911813 6357 leader_election.cc:302] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a70724bcb0d4d1fb6d7c243c5009a84; no voters:
I20260808 00:41:43.912026 6357 leader_election.cc:288] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:43.912062 6363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:43.912235 6363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [term 1 LEADER]: Becoming Leader. State: Replica: 4a70724bcb0d4d1fb6d7c243c5009a84, State: Running, Role: LEADER
I20260808 00:41:43.912271 6357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:43.912386 6363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } }
I20260808 00:41:43.912694 6363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a70724bcb0d4d1fb6d7c243c5009a84. Latest consensus state: current_term: 1 leader_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } } }
I20260808 00:41:43.912848 6366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a70724bcb0d4d1fb6d7c243c5009a84" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43917 } } }
I20260808 00:41:43.914259 6366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:43.913818 6363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84 [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:43.914052 6387 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:43.914486 6387 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:43.914549 6392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:43.914727 6392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:43.915113 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:43.909053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:44741 (local address 127.0.0.1:43917)
0808 00:41:43.909244 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:43.909254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:43.909313 (+ 59us) server_negotiation.cc:407] Connection header received
0808 00:41:43.910137 (+ 824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:43.910144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:43.910194 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:43.910315 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:43.910524 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:43.911791 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:43.912924 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:43.913520 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:43.913627 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:43.914391 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:43.914395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:43.914395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:43.914595 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:43.914633 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:43.914648 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:43.914736 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:43.914865 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:41:43.914893 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260808 00:41:43.916198 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:43.916312 6392 catalog_manager.cc:1347] Generated new cluster ID: f63f4db6d4f44c23ab9f0f37a329a876
I20260808 00:41:43.916348 6392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:43.924849 6392 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:43.925101 6361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:43.921554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:36967 (local address 127.0.0.1:43917)
0808 00:41:43.921594 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:43.921596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:43.921744 (+ 148us) server_negotiation.cc:407] Connection header received
0808 00:41:43.921791 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:43.921792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:43.921802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:43.921846 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:43.921989 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:43.922692 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:43.923724 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:43.924037 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:43.924100 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:43.924741 (+ 641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:43.924743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:43.924743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:43.924748 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:43.924762 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:43.924767 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:43.924838 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:43.925011 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 00:41:43.925023 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":8,"server-negotiator.queue_time_us":21}
I20260808 00:41:43.925509 6392 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:43.934345 6392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a70724bcb0d4d1fb6d7c243c5009a84: Generated new TSK 0
I20260808 00:41:43.934510 6392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:43.936523 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6266
W20260808 00:41:43.950614 6245 connection.cc:535] client connection to 127.0.0.1:43917 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43917): Connection reset by peer (error 104)
2026-08-08T00:41:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1445 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:43 dist-test-slave-gjw7 krb5kdc[6406](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:43 dist-test-slave-gjw7 krb5kdc[6406](info): set up 2 sockets
Aug 08 00:41:43 dist-test-slave-gjw7 krb5kdc[6406](info): commencing operation
krb5kdc: starting...
W20260808 00:41:45.111647 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:45 dist-test-slave-gjw7 krb5kdc[6406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149705, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 00:41:45.229772 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41261
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:45.313480 6428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:45.313690 6428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:45.313714 6428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:45.314862 6428 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:45.314909 6428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:45.314925 6428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:45.314939 6428 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:45.314954 6428 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:45.316491 6428 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41261
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:45.316699 6428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:45.316885 6428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:45.321532 6439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:45.321540 6436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:45.321614 6428 server_base.cc:1034] running on GCE node
W20260808 00:41:45.321549 6442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:45.322036 6428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:45.322306 6428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:45.323453 6428 hybrid_clock.cc:648] HybridClock initialized: now 1786149705323437 us; error 31 us; skew 500 ppm
Aug 08 00:41:45 dist-test-slave-gjw7 krb5kdc[6406](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149705, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:45.325850 6428 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 00:41:45.326232 6428 webserver.cc:460] Webserver started at http://127.0.0.1:37793/ using document root <none> and password file <none>
I20260808 00:41:45.326421 6428 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:45.326458 6428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:45.326550 6428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:45.327548 6428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "a519e9438ccc45f19fdcd57f679c698a"
format_stamp: "Formatted at 2026-08-08 00:41:45 on dist-test-slave-gjw7"
server_key: "ddd1c0399dc1c1aae3dfbcb52d7762b2"
server_key_iv: "d346382649e9a2eb54448390a8c80b9a"
server_key_version: "encryptionkey@0"
I20260808 00:41:45.327924 6428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "a519e9438ccc45f19fdcd57f679c698a"
format_stamp: "Formatted at 2026-08-08 00:41:45 on dist-test-slave-gjw7"
server_key: "ddd1c0399dc1c1aae3dfbcb52d7762b2"
server_key_iv: "d346382649e9a2eb54448390a8c80b9a"
server_key_version: "encryptionkey@0"
I20260808 00:41:45.329078 6428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:45.329735 6451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:45.329897 6428 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:45.329978 6428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "a519e9438ccc45f19fdcd57f679c698a"
format_stamp: "Formatted at 2026-08-08 00:41:45 on dist-test-slave-gjw7"
server_key: "ddd1c0399dc1c1aae3dfbcb52d7762b2"
server_key_iv: "d346382649e9a2eb54448390a8c80b9a"
server_key_version: "encryptionkey@0"
I20260808 00:41:45.330034 6428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:45.333097 6428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:45.334967 6428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:45.335069 6428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:45.338879 6428 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41261
I20260808 00:41:45.338917 6538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41261 every 8 connection(s)
I20260808 00:41:45.339197 6428 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:45.340019 6540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:45.342655 6540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: Bootstrap starting.
I20260808 00:41:45.343356 6540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:45.343685 6540 log.cc:824] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:45.344444 6540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: No bootstrap required, opened a new log
I20260808 00:41:45.344588 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6428
I20260808 00:41:45.344686 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:45.344856 1856 external_mini_cluster.cc:1428] Setting key f7fbea13b7ebeb80c9f5969f075d4898
I20260808 00:41:45.346560 6540 raft_consensus.cc:348] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260808 00:41:45.346733 6540 raft_consensus.cc:374] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:45.346772 6540 raft_consensus.cc:729] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a519e9438ccc45f19fdcd57f679c698a, State: Initialized, Role: FOLLOWER
I20260808 00:41:45.346890 6540 consensus_queue.cc:260] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260808 00:41:45.346979 6540 raft_consensus.cc:388] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:45.347019 6540 raft_consensus.cc:482] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:45.347060 6540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:45.347779 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:45.345337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:60719 (local address 127.0.0.1:41261)
0808 00:41:45.345500 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:45.345509 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:45.345543 (+ 34us) server_negotiation.cc:407] Connection header received
0808 00:41:45.346523 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:45.346532 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:45.346571 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:45.346736 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:45.347319 (+ 583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:45.347323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:45.347324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:45.347328 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:45.347378 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:45.347393 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:45.347471 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:45.347509 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 00:41:45.347554 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260808 00:41:45.348105 6540 raft_consensus.cc:504] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260808 00:41:45.348213 6540 leader_election.cc:302] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a519e9438ccc45f19fdcd57f679c698a; no voters:
I20260808 00:41:45.348385 6540 leader_election.cc:288] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:45.348416 6545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:45.348600 6545 raft_consensus.cc:686] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [term 1 LEADER]: Becoming Leader. State: Replica: a519e9438ccc45f19fdcd57f679c698a, State: Running, Role: LEADER
I20260808 00:41:45.348624 6540 sys_catalog.cc:564] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:45.348711 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:45.348695 6545 consensus_queue.cc:237] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } }
I20260808 00:41:45.349054 6545 sys_catalog.cc:455] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a519e9438ccc45f19fdcd57f679c698a. Latest consensus state: current_term: 1 leader_uuid: "a519e9438ccc45f19fdcd57f679c698a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } } }
I20260808 00:41:45.349131 6545 sys_catalog.cc:458] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:45.349306 6547 sys_catalog.cc:455] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a519e9438ccc45f19fdcd57f679c698a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a519e9438ccc45f19fdcd57f679c698a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41261 } } }
I20260808 00:41:45.349370 6547 sys_catalog.cc:458] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:45.350401 6574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:45.350490 6574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:45.350579 6557 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:45.350785 6557 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:45.351346 6544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:45.350376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.206:49581 (local address 127.0.0.1:41261)
0808 00:41:45.350411 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:45.350414 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:45.350530 (+ 116us) server_negotiation.cc:407] Connection header received
0808 00:41:45.350608 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:45.350608 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:45.350614 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:45.350662 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:45.351152 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:45.351153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:45.351153 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:45.351157 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:45.351171 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:45.351175 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:45.351220 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:45.351266 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 00:41:45.351278 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:45.352276 6557 catalog_manager.cc:1347] Generated new cluster ID: f075609fcbe044b9864c32b6a4445199
I20260808 00:41:45.352316 6557 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:45.381954 6557 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:45.382143 6557 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:45.388526 6557 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a519e9438ccc45f19fdcd57f679c698a: Generated new TSK 0
I20260808 00:41:45.388731 6557 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:45.391252 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6428
2026-08-08T00:41:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1453 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:45 dist-test-slave-gjw7 krb5kdc[6584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:45 dist-test-slave-gjw7 krb5kdc[6584](info): set up 2 sockets
Aug 08 00:41:45 dist-test-slave-gjw7 krb5kdc[6584](info): commencing operation
krb5kdc: starting...
W20260808 00:41:46.566977 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:46.684005 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:39761
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:46.764671 6600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:46.764873 6600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:46.764902 6600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:46.766026 6600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:46.766067 6600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:46.766085 6600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:46.766100 6600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:46.766115 6600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:46.767578 6600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:39761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:46.767827 6600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:46.768035 6600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:46.772712 6610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:46.772717 6608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:46.772786 6600 server_base.cc:1034] running on GCE node
W20260808 00:41:46.772725 6612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:46.773307 6600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:46.773613 6600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:46.774775 6600 hybrid_clock.cc:648] HybridClock initialized: now 1786149706774750 us; error 35 us; skew 500 ppm
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149706, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:46.777343 6600 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:46.777777 6600 webserver.cc:460] Webserver started at http://127.1.208.62:41611/ using document root <none> and password file <none>
I20260808 00:41:46.778000 6600 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:46.778071 6600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:46.778214 6600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:46.779273 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a"
format_stamp: "Formatted at 2026-08-08 00:41:46 on dist-test-slave-gjw7"
server_key: "c900c835ab14e715a65b799c4d190ce1"
server_key_iv: "49b97406d8523ebd9ca87d69d93e9409"
server_key_version: "encryptionkey@0"
I20260808 00:41:46.779636 6600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a"
format_stamp: "Formatted at 2026-08-08 00:41:46 on dist-test-slave-gjw7"
server_key: "c900c835ab14e715a65b799c4d190ce1"
server_key_iv: "49b97406d8523ebd9ca87d69d93e9409"
server_key_version: "encryptionkey@0"
I20260808 00:41:46.780974 6600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:46.781762 6626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:46.781913 6600 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:46.781999 6600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a"
format_stamp: "Formatted at 2026-08-08 00:41:46 on dist-test-slave-gjw7"
server_key: "c900c835ab14e715a65b799c4d190ce1"
server_key_iv: "49b97406d8523ebd9ca87d69d93e9409"
server_key_version: "encryptionkey@0"
I20260808 00:41:46.782060 6600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:46.794620 6600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:46.796275 6600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:46.796404 6600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:46.800374 6600 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:39761
I20260808 00:41:46.800397 6714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:39761 every 8 connection(s)
I20260808 00:41:46.800740 6600 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:46.801604 6715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:46.803819 6715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a: Bootstrap starting.
I20260808 00:41:46.804364 6715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:46.804628 6715 log.cc:824] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:46.805274 6715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a: No bootstrap required, opened a new log
I20260808 00:41:46.806911 6715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } }
I20260808 00:41:46.807046 6715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:46.807068 6715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bdec7033f8a4d8d8bd5954a5a8d299a, State: Initialized, Role: FOLLOWER
I20260808 00:41:46.807204 6715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } }
I20260808 00:41:46.807281 6715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:46.807315 6715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:46.807372 6715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:46.808158 6715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } }
I20260808 00:41:46.808250 6715 leader_election.cc:302] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bdec7033f8a4d8d8bd5954a5a8d299a; no voters:
I20260808 00:41:46.808424 6715 leader_election.cc:288] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:46.808442 6721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:46.808508 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6600
I20260808 00:41:46.808554 6721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [term 1 LEADER]: Becoming Leader. State: Replica: 8bdec7033f8a4d8d8bd5954a5a8d299a, State: Running, Role: LEADER
I20260808 00:41:46.808625 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:46.808629 6721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } }
I20260808 00:41:46.808682 6715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:46.808801 1856 external_mini_cluster.cc:1428] Setting key e32ae21f813ecd3f8c7153b6673326cb
I20260808 00:41:46.808956 6721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bdec7033f8a4d8d8bd5954a5a8d299a. Latest consensus state: current_term: 1 leader_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } } }
I20260808 00:41:46.809028 6721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:46.808964 6722 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bdec7033f8a4d8d8bd5954a5a8d299a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 39761 } } }
I20260808 00:41:46.809082 6722 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:46.809831 6726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:46.810392 6726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:46.811932 6726 catalog_manager.cc:1347] Generated new cluster ID: 6779e5ff03b3445cb86017bec647a2db
I20260808 00:41:46.811983 6726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:46.820609 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:46.809355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34697 (local address 127.1.208.62:39761)
0808 00:41:46.809671 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:46.809680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:46.809720 (+ 40us) server_negotiation.cc:407] Connection header received
0808 00:41:46.809864 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:46.809871 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:46.809932 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:46.810149 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:46.810964 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:46.812347 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:46.813451 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:46.814086 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:46.814206 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:46.816943 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:46.816948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:46.816948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:46.816960 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:46.818979 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:46.819414 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:46.819417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:46.819417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:46.819516 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:46.819732 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:46.819733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:46.819733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:46.820018 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:46.820165 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:46.820289 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 00:41:46.820383 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":70,"threads_started":1}
I20260808 00:41:46.821856 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:46.824129 6726 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:46.824774 6726 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:46.830384 6735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:46.823288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37425 (local address 127.1.208.62:39761)
0808 00:41:46.823321 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:46.823323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:46.823449 (+ 126us) server_negotiation.cc:407] Connection header received
0808 00:41:46.823503 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:46.823504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:46.823515 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:46.823566 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:46.824177 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:46.824881 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:46.825887 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:46.826201 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:46.826261 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:46.827520 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:46.827521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:46.827521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:46.827527 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:46.828954 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:46.829373 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:46.829374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:46.829374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:46.829423 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:46.829615 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:46.829616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:46.829616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:46.829724 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:46.830195 (+ 471us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:46.830240 (+ 45us) server_negotiation.cc:299] Negotiation successful
0808 00:41:46.830298 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:46.838137 6726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bdec7033f8a4d8d8bd5954a5a8d299a: Generated new TSK 0
I20260808 00:41:46.838341 6726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:46.841584 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6600
W20260808 00:41:46.856232 6581 connection.cc:535] client connection to 127.1.208.62:39761 recv error: Network error: failed to read from TLS socket (remote: 127.1.208.62:39761): Connection reset by peer (error 104)
2026-08-08T00:41:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6758](info): set up 2 sockets
Aug 08 00:41:46 dist-test-slave-gjw7 krb5kdc[6758](info): commencing operation
krb5kdc: starting...
W20260808 00:41:48.012970 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:48 dist-test-slave-gjw7 krb5kdc[6758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:48.133388 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40073
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:40073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:48.217767 6780 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:48.218010 6780 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:48.218045 6780 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:48.219213 6780 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:48.219259 6780 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:48.219280 6780 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:48.219298 6780 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:48.219318 6780 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:48.220959 6780 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:40073
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:40073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6780
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:48.221236 6780 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:48.221486 6780 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:48.226205 6789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:48.226176 6788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:48.226181 6793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:48.226369 6780 server_base.cc:1034] running on GCE node
I20260808 00:41:48.226738 6780 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:48.227046 6780 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:48.228188 6780 hybrid_clock.cc:648] HybridClock initialized: now 1786149708228172 us; error 32 us; skew 500 ppm
Aug 08 00:41:48 dist-test-slave-gjw7 krb5kdc[6758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149708, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:48.233969 6780 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:48.234370 6780 webserver.cc:460] Webserver started at http://127.1.208.62:39023/ using document root <none> and password file <none>
I20260808 00:41:48.234546 6780 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:48.234583 6780 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:48.234673 6780 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:48.235673 6780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "f15248d00a7948f286c5cbf84637975b"
format_stamp: "Formatted at 2026-08-08 00:41:48 on dist-test-slave-gjw7"
server_key: "8c9062a21bf1dafb3c2a030e57157b88"
server_key_iv: "54c9ead5837694057df704148c7d5087"
server_key_version: "encryptionkey@0"
I20260808 00:41:48.235991 6780 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "f15248d00a7948f286c5cbf84637975b"
format_stamp: "Formatted at 2026-08-08 00:41:48 on dist-test-slave-gjw7"
server_key: "8c9062a21bf1dafb3c2a030e57157b88"
server_key_iv: "54c9ead5837694057df704148c7d5087"
server_key_version: "encryptionkey@0"
I20260808 00:41:48.237172 6780 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260808 00:41:48.237939 6808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:48.238113 6780 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:48.238196 6780 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "f15248d00a7948f286c5cbf84637975b"
format_stamp: "Formatted at 2026-08-08 00:41:48 on dist-test-slave-gjw7"
server_key: "8c9062a21bf1dafb3c2a030e57157b88"
server_key_iv: "54c9ead5837694057df704148c7d5087"
server_key_version: "encryptionkey@0"
I20260808 00:41:48.238281 6780 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:48.254719 6780 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:48.256515 6780 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:48.256631 6780 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:48.260586 6780 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:40073
I20260808 00:41:48.260622 6921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:40073 every 8 connection(s)
I20260808 00:41:48.260931 6780 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:48.261823 6922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:48.264175 6922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: Bootstrap starting.
I20260808 00:41:48.264773 6922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:48.265033 6922 log.cc:824] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:48.265765 6922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: No bootstrap required, opened a new log
I20260808 00:41:48.267433 6922 raft_consensus.cc:348] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } }
I20260808 00:41:48.267572 6922 raft_consensus.cc:374] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:48.267604 6922 raft_consensus.cc:729] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f15248d00a7948f286c5cbf84637975b, State: Initialized, Role: FOLLOWER
I20260808 00:41:48.267735 6922 consensus_queue.cc:260] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } }
I20260808 00:41:48.267815 6922 raft_consensus.cc:388] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:48.267855 6922 raft_consensus.cc:482] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:48.267907 6922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:48.267969 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6780
I20260808 00:41:48.268090 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:48.268309 1856 external_mini_cluster.cc:1428] Setting key a6ba488831dbf0d1160029247d3f51a2
I20260808 00:41:48.268849 6922 raft_consensus.cc:504] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } }
I20260808 00:41:48.268957 6922 leader_election.cc:302] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f15248d00a7948f286c5cbf84637975b; no voters:
I20260808 00:41:48.269119 6922 leader_election.cc:288] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:48.269152 6931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:48.269256 6931 raft_consensus.cc:686] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [term 1 LEADER]: Becoming Leader. State: Replica: f15248d00a7948f286c5cbf84637975b, State: Running, Role: LEADER
I20260808 00:41:48.269359 6931 consensus_queue.cc:237] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } }
I20260808 00:41:48.269414 6922 sys_catalog.cc:564] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:48.269727 6933 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f15248d00a7948f286c5cbf84637975b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } } }
I20260808 00:41:48.269843 6933 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:48.269721 6931 sys_catalog.cc:455] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f15248d00a7948f286c5cbf84637975b. Latest consensus state: current_term: 1 leader_uuid: "f15248d00a7948f286c5cbf84637975b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f15248d00a7948f286c5cbf84637975b" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 40073 } } }
I20260808 00:41:48.270061 6931 sys_catalog.cc:458] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:48.271230 6951 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:48.271287 6951 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:48.271329 6941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:48.271467 6941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:48.274040 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:48.268766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42783 (local address 127.1.208.62:40073)
0808 00:41:48.268965 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:48.268974 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:48.269007 (+ 33us) server_negotiation.cc:407] Connection header received
0808 00:41:48.269122 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:48.269129 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:48.269184 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:48.269582 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:48.269696 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:48.271137 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:48.271991 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:48.272561 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:48.272676 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:48.273368 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:48.273373 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:48.273373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:48.273536 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:48.273566 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:48.273577 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:48.273642 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:48.273776 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 00:41:48.273817 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260808 00:41:48.274979 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:48.277705 6941 catalog_manager.cc:1347] Generated new cluster ID: be92f2637fb441e4ab9d6faedd1be3f9
I20260808 00:41:48.277761 6941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:48.279279 6930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:48.276250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52023 (local address 127.1.208.62:40073)
0808 00:41:48.276284 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:48.276286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:48.276411 (+ 125us) server_negotiation.cc:407] Connection header received
0808 00:41:48.276474 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:48.276475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:48.276482 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:48.276527 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:48.276663 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:48.277436 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:48.278227 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:48.278451 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:48.278504 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:48.279075 (+ 571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:48.279076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:48.279077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:48.279081 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:48.279095 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:48.279099 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:48.279147 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:48.279201 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 00:41:48.279212 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:41:48.290066 6941 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:48.290581 6941 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:48.295089 6941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f15248d00a7948f286c5cbf84637975b: Generated new TSK 0
I20260808 00:41:48.295260 6941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:48.300227 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6780
W20260808 00:41:48.314936 6756 connection.cc:535] client connection to 127.1.208.62:40073 recv error: Network error: failed to read from TLS socket (remote: 127.1.208.62:40073): Connection reset by peer (error 104)
2026-08-08T00:41:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:48 dist-test-slave-gjw7 krb5kdc[6969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:48 dist-test-slave-gjw7 krb5kdc[6969](info): set up 2 sockets
Aug 08 00:41:48 dist-test-slave-gjw7 krb5kdc[6969](info): commencing operation
krb5kdc: starting...
W20260808 00:41:49.486860 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:49 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:49.606185 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43799
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:43827
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:43799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:49.688913 6994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:49.689105 6994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:49.689127 6994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:49.690258 6994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:49.690299 6994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:49.690312 6994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:49.690326 6994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:49.690341 6994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:49.691855 6994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:43827
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:43799
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:43799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6994
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:49.692086 6994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:49.692291 6994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:49.696925 7002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:49.697011 6994 server_base.cc:1034] running on GCE node
W20260808 00:41:49.696949 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:49.696925 7003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:49.697496 6994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:49.697824 6994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:49.699035 6994 hybrid_clock.cc:648] HybridClock initialized: now 1786149709698995 us; error 36 us; skew 500 ppm
Aug 08 00:41:49 dist-test-slave-gjw7 krb5kdc[6969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149709, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:49.701509 6994 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:49.701973 6994 webserver.cc:460] Webserver started at http://127.1.208.62:43413/ using document root <none> and password file <none>
I20260808 00:41:49.702193 6994 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:49.702243 6994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:49.702368 6994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:49.703392 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "654569275b9f49c1b68fbdf95c4e7a22"
format_stamp: "Formatted at 2026-08-08 00:41:49 on dist-test-slave-gjw7"
server_key: "90eda37bd5e463152e03c1e836227bb7"
server_key_iv: "bbd9f68f7fdcf013d5c66352de5b4666"
server_key_version: "encryptionkey@0"
I20260808 00:41:49.703727 6994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "654569275b9f49c1b68fbdf95c4e7a22"
format_stamp: "Formatted at 2026-08-08 00:41:49 on dist-test-slave-gjw7"
server_key: "90eda37bd5e463152e03c1e836227bb7"
server_key_iv: "bbd9f68f7fdcf013d5c66352de5b4666"
server_key_version: "encryptionkey@0"
I20260808 00:41:49.704963 6994 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:49.705657 7018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:49.705783 6994 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:49.705868 6994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "654569275b9f49c1b68fbdf95c4e7a22"
format_stamp: "Formatted at 2026-08-08 00:41:49 on dist-test-slave-gjw7"
server_key: "90eda37bd5e463152e03c1e836227bb7"
server_key_iv: "bbd9f68f7fdcf013d5c66352de5b4666"
server_key_version: "encryptionkey@0"
I20260808 00:41:49.705950 6994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:49.709127 6994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:49.710881 6994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:49.711045 6994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:49.714937 6994 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:43799
I20260808 00:41:49.714975 7101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:43799 every 8 connection(s)
I20260808 00:41:49.715268 6994 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:49.716259 7103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:49.718530 7103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22: Bootstrap starting.
I20260808 00:41:49.719056 7103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:49.719285 7103 log.cc:824] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:49.719889 7103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22: No bootstrap required, opened a new log
I20260808 00:41:49.721269 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 6994
I20260808 00:41:49.721418 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:49.721573 1856 external_mini_cluster.cc:1428] Setting key bac78951ffce493f0429ebc21c08519d
I20260808 00:41:49.721529 7103 raft_consensus.cc:348] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } }
I20260808 00:41:49.721675 7103 raft_consensus.cc:374] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:49.721750 7103 raft_consensus.cc:729] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654569275b9f49c1b68fbdf95c4e7a22, State: Initialized, Role: FOLLOWER
I20260808 00:41:49.721905 7103 consensus_queue.cc:260] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } }
I20260808 00:41:49.722013 7103 raft_consensus.cc:388] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:49.722059 7103 raft_consensus.cc:482] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:49.722143 7103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:49.723069 7103 raft_consensus.cc:504] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } }
I20260808 00:41:49.723181 7103 leader_election.cc:302] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 654569275b9f49c1b68fbdf95c4e7a22; no voters:
I20260808 00:41:49.723343 7103 leader_election.cc:288] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:49.723378 7109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:49.723628 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:49.722051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41123 (local address 127.1.208.62:43799)
0808 00:41:49.722234 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:49.722244 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:49.722270 (+ 26us) server_negotiation.cc:407] Connection header received
0808 00:41:49.722369 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:49.722374 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:49.722397 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:49.722513 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:49.723210 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:49.723214 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:49.723214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:49.723218 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:49.723263 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:49.723278 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:49.723332 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:49.723397 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:41:49.723437 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260808 00:41:49.723676 7109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [term 1 LEADER]: Becoming Leader. State: Replica: 654569275b9f49c1b68fbdf95c4e7a22, State: Running, Role: LEADER
I20260808 00:41:49.723675 7103 sys_catalog.cc:564] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:49.723803 7109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } }
I20260808 00:41:49.724205 7110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "654569275b9f49c1b68fbdf95c4e7a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } } }
I20260808 00:41:49.724292 7110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:49.724483 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:49.724514 7111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 654569275b9f49c1b68fbdf95c4e7a22. Latest consensus state: current_term: 1 leader_uuid: "654569275b9f49c1b68fbdf95c4e7a22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "654569275b9f49c1b68fbdf95c4e7a22" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 43799 } } }
I20260808 00:41:49.724570 7111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:49.724792 7121 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:49.725255 7121 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:49.727236 7108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:49.726004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49433 (local address 127.1.208.62:43799)
0808 00:41:49.726060 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:49.726062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:49.726442 (+ 380us) server_negotiation.cc:407] Connection header received
0808 00:41:49.726501 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:49.726502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:49.726507 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:49.726550 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:49.727049 (+ 499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:49.727050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:49.727050 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:49.727054 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:49.727066 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:49.727070 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:49.727115 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:49.727163 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 00:41:49.727175 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:41:49.732252 7121 catalog_manager.cc:1347] Generated new cluster ID: 9859a7c6acc44780910bdfc475c40a05
I20260808 00:41:49.732314 7121 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:49.745476 7121 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:49.745646 7121 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:49.750433 7121 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 654569275b9f49c1b68fbdf95c4e7a22: Generated new TSK 0
I20260808 00:41:49.750603 7121 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:49.755671 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 6994
2026-08-08T00:41:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1452 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:49 dist-test-slave-gjw7 krb5kdc[7158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:49 dist-test-slave-gjw7 krb5kdc[7158](info): set up 2 sockets
Aug 08 00:41:49 dist-test-slave-gjw7 krb5kdc[7158](info): commencing operation
krb5kdc: starting...
W20260808 00:41:50.918349 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:50 dist-test-slave-gjw7 krb5kdc[7158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:51.035941 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34833
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:38021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:34833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:51.117116 7189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:51.117354 7189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:51.117388 7189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:51.118525 7189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:51.118563 7189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:51.118577 7189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:51.118587 7189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:51.118598 7189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:51.120034 7189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:38021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:34833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:34833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:51.120268 7189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:51.120473 7189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:51.125015 7199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:51.125030 7198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:51.125029 7201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:51.125389 7189 server_base.cc:1034] running on GCE node
I20260808 00:41:51.125643 7189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:51.125932 7189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:51.127072 7189 hybrid_clock.cc:648] HybridClock initialized: now 1786149711127063 us; error 33 us; skew 500 ppm
Aug 08 00:41:51 dist-test-slave-gjw7 krb5kdc[7158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149711, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:51.129487 7189 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:51.129923 7189 webserver.cc:460] Webserver started at http://127.1.208.62:35611/ using document root <none> and password file <none>
I20260808 00:41:51.130117 7189 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:51.130163 7189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:51.130287 7189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:51.131292 7189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef"
format_stamp: "Formatted at 2026-08-08 00:41:51 on dist-test-slave-gjw7"
server_key: "e62f33187d3829bd8cd73dc779f5d351"
server_key_iv: "55c16c641220ca4fcecf64e8a32cdfcd"
server_key_version: "encryptionkey@0"
I20260808 00:41:51.131618 7189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef"
format_stamp: "Formatted at 2026-08-08 00:41:51 on dist-test-slave-gjw7"
server_key: "e62f33187d3829bd8cd73dc779f5d351"
server_key_iv: "55c16c641220ca4fcecf64e8a32cdfcd"
server_key_version: "encryptionkey@0"
I20260808 00:41:51.132869 7189 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:51.133548 7210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:51.133683 7189 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:51.133766 7189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef"
format_stamp: "Formatted at 2026-08-08 00:41:51 on dist-test-slave-gjw7"
server_key: "e62f33187d3829bd8cd73dc779f5d351"
server_key_iv: "55c16c641220ca4fcecf64e8a32cdfcd"
server_key_version: "encryptionkey@0"
I20260808 00:41:51.133837 7189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:51.152608 7189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:51.154776 7189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:51.154922 7189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:51.158921 7189 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:34833
I20260808 00:41:51.158957 7303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:34833 every 8 connection(s)
I20260808 00:41:51.159286 7189 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:51.160116 7305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:51.160635 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7189
I20260808 00:41:51.160733 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:51.160903 1856 external_mini_cluster.cc:1428] Setting key cc05193257120397a6fd17ed53dff97b
I20260808 00:41:51.163390 7305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef: Bootstrap starting.
I20260808 00:41:51.164088 7305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:51.164402 7305 log.cc:824] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:51.165087 7305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef: No bootstrap required, opened a new log
I20260808 00:41:51.167232 7305 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } }
I20260808 00:41:51.167411 7305 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:51.167445 7305 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27cbe927e08f4e8f9d9ea3beabcea2ef, State: Initialized, Role: FOLLOWER
Aug 08 00:41:51 dist-test-slave-gjw7 krb5kdc[7158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:51.167567 7305 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } }
I20260808 00:41:51.167649 7305 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:51.167687 7305 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:51.167733 7305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:51.168349 7305 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } }
I20260808 00:41:51.168458 7305 leader_election.cc:302] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27cbe927e08f4e8f9d9ea3beabcea2ef; no voters:
I20260808 00:41:51.168618 7305 leader_election.cc:288] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:51.168663 7313 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:51.168772 7313 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [term 1 LEADER]: Becoming Leader. State: Replica: 27cbe927e08f4e8f9d9ea3beabcea2ef, State: Running, Role: LEADER
I20260808 00:41:51.168853 7313 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } }
I20260808 00:41:51.168883 7305 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:51.169833 7318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27cbe927e08f4e8f9d9ea3beabcea2ef. Latest consensus state: current_term: 1 leader_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } } }
I20260808 00:41:51.169858 7316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27cbe927e08f4e8f9d9ea3beabcea2ef" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 34833 } } }
I20260808 00:41:51.169926 7318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:51.169931 7316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:51.170133 7334 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:51.170616 7334 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:51.171806 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:51.161416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46213 (local address 127.1.208.62:34833)
0808 00:41:51.161581 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:51.161591 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:51.161626 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:41:51.162156 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:51.162166 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:51.162246 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:51.162389 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:51.162964 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:51.164246 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:51.164996 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:51.165585 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:51.165694 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:51.168048 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:51.168052 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:51.168052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:51.168064 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:51.170100 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:51.170528 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:51.170531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:51.170531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:51.170653 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:51.170879 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:51.170880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:51.170880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:51.171136 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:51.171233 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:51.171495 (+ 262us) server_negotiation.cc:299] Negotiation successful
0808 00:41:51.171632 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260808 00:41:51.172508 7334 catalog_manager.cc:1347] Generated new cluster ID: d90e1bad656844b08979d301eed8d814
I20260808 00:41:51.172559 7334 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:51.172936 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:51.183645 7334 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:51.184162 7309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:51.174333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51169 (local address 127.1.208.62:34833)
0808 00:41:51.174363 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:51.174364 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:51.177445 (+ 3081us) server_negotiation.cc:407] Connection header received
0808 00:41:51.177479 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:51.177480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:51.177490 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:51.177561 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:51.178106 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:51.178808 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:51.179757 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:51.180071 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:51.180143 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:51.181386 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:51.181387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:51.181388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:51.181395 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:51.182884 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:51.183258 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:51.183259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:51.183259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:51.183315 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:51.183551 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:51.183552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:51.183552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:51.183685 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:51.183787 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:51.183994 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 00:41:51.184073 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260808 00:41:51.184281 7334 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:51.188931 7334 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27cbe927e08f4e8f9d9ea3beabcea2ef: Generated new TSK 0
I20260808 00:41:51.189103 7334 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:51.191022 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7189
2026-08-08T00:41:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:51 dist-test-slave-gjw7 krb5kdc[7361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:51 dist-test-slave-gjw7 krb5kdc[7361](info): set up 2 sockets
Aug 08 00:41:51 dist-test-slave-gjw7 krb5kdc[7361](info): commencing operation
krb5kdc: starting...
W20260808 00:41:52.358955 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:52 dist-test-slave-gjw7 krb5kdc[7361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:52.475414 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38553
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:46693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:38553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:52.555773 7387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:52.555989 7387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:52.556011 7387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:52.557096 7387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:52.557126 7387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:52.557138 7387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:52.557150 7387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:52.557160 7387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:52.558586 7387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:46693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:38553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:38553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:52.558772 7387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:52.558939 7387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:52.563490 7397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:52.563467 7399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:52.563493 7394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:52.563607 7387 server_base.cc:1034] running on GCE node
I20260808 00:41:52.563957 7387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:52.564249 7387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:52.565385 7387 hybrid_clock.cc:648] HybridClock initialized: now 1786149712565370 us; error 32 us; skew 500 ppm
Aug 08 00:41:52 dist-test-slave-gjw7 krb5kdc[7361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149712, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:52.567878 7387 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:52.568308 7387 webserver.cc:460] Webserver started at http://127.1.208.62:46229/ using document root <none> and password file <none>
I20260808 00:41:52.568507 7387 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:52.568554 7387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:52.568677 7387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:52.569763 7387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "c504c17940c54727853b2aa32f8ad528"
format_stamp: "Formatted at 2026-08-08 00:41:52 on dist-test-slave-gjw7"
server_key: "50b83e7894c99b760a6e12995a1c8af3"
server_key_iv: "9dc85476e106574267407431943ede77"
server_key_version: "encryptionkey@0"
I20260808 00:41:52.570092 7387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "c504c17940c54727853b2aa32f8ad528"
format_stamp: "Formatted at 2026-08-08 00:41:52 on dist-test-slave-gjw7"
server_key: "50b83e7894c99b760a6e12995a1c8af3"
server_key_iv: "9dc85476e106574267407431943ede77"
server_key_version: "encryptionkey@0"
I20260808 00:41:52.571346 7387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:52.572005 7408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:52.572157 7387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:52.572239 7387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "c504c17940c54727853b2aa32f8ad528"
format_stamp: "Formatted at 2026-08-08 00:41:52 on dist-test-slave-gjw7"
server_key: "50b83e7894c99b760a6e12995a1c8af3"
server_key_iv: "9dc85476e106574267407431943ede77"
server_key_version: "encryptionkey@0"
I20260808 00:41:52.572325 7387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:52.585593 7387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:52.587282 7387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:52.587409 7387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:52.591223 7387 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:38553
I20260808 00:41:52.591257 7498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:38553 every 8 connection(s)
I20260808 00:41:52.591562 7387 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:52.592368 7499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:52.594569 7499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: Bootstrap starting.
I20260808 00:41:52.595120 7499 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:52.595371 7499 log.cc:824] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:52.596006 7499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: No bootstrap required, opened a new log
I20260808 00:41:52.597616 7499 raft_consensus.cc:348] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } }
I20260808 00:41:52.597748 7499 raft_consensus.cc:374] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:52.597786 7499 raft_consensus.cc:729] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c504c17940c54727853b2aa32f8ad528, State: Initialized, Role: FOLLOWER
I20260808 00:41:52.597929 7499 consensus_queue.cc:260] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } }
I20260808 00:41:52.597990 7499 raft_consensus.cc:388] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:52.598013 7499 raft_consensus.cc:482] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:52.598058 7499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:52.598847 7499 raft_consensus.cc:504] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } }
I20260808 00:41:52.598935 7499 leader_election.cc:302] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c504c17940c54727853b2aa32f8ad528; no voters:
I20260808 00:41:52.599079 7499 leader_election.cc:288] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:52.599118 7503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:52.599261 7503 raft_consensus.cc:686] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [term 1 LEADER]: Becoming Leader. State: Replica: c504c17940c54727853b2aa32f8ad528, State: Running, Role: LEADER
I20260808 00:41:52.599308 7499 sys_catalog.cc:564] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:52.599413 7503 consensus_queue.cc:237] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } }
I20260808 00:41:52.599735 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7387
I20260808 00:41:52.599763 7504 sys_catalog.cc:455] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c504c17940c54727853b2aa32f8ad528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } } }
I20260808 00:41:52.599841 7504 sys_catalog.cc:458] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:52.599861 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:52.600059 1856 external_mini_cluster.cc:1428] Setting key 7a921452bee3b15c204438b37036a0d9
I20260808 00:41:52.601025 7505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c504c17940c54727853b2aa32f8ad528. Latest consensus state: current_term: 1 leader_uuid: "c504c17940c54727853b2aa32f8ad528" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c504c17940c54727853b2aa32f8ad528" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 38553 } } }
I20260808 00:41:52.601188 7505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528 [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:52.601212 7535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:52.601317 7535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:52.601449 7519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:52.601610 7519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:52.602859 7519 catalog_manager.cc:1347] Generated new cluster ID: d55ae2601b9442198480e57668f5b327
I20260808 00:41:52.602906 7519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:52.605926 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:52.600572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45559 (local address 127.1.208.62:38553)
0808 00:41:52.600764 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:52.600774 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:52.600806 (+ 32us) server_negotiation.cc:407] Connection header received
0808 00:41:52.600933 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:52.600940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:52.600981 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:52.601097 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:52.601442 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:52.602785 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:52.603882 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:52.604476 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:52.604588 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:52.605232 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:52.605236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:52.605236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:52.605459 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:52.605501 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:52.605516 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:52.605589 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:52.605651 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:41:52.605714 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260808 00:41:52.606834 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:52.613024 7519 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:52.613705 7519 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:52.616006 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:52.609570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51229 (local address 127.1.208.62:38553)
0808 00:41:52.609603 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:52.609604 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:52.612760 (+ 3156us) server_negotiation.cc:407] Connection header received
0808 00:41:52.612808 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:52.612810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:52.612820 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:52.612878 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:52.613051 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:52.613755 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:52.614790 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:52.615078 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:52.615133 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:52.615734 (+ 601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:52.615735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:52.615735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:52.615741 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:52.615756 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:52.615760 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:52.615815 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:52.615877 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 00:41:52.615934 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:52.618642 7519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c504c17940c54727853b2aa32f8ad528: Generated new TSK 0
I20260808 00:41:52.618815 7519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:52.621665 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7387
2026-08-08T00:41:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:52 dist-test-slave-gjw7 krb5kdc[7553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:52 dist-test-slave-gjw7 krb5kdc[7553](info): set up 2 sockets
Aug 08 00:41:52 dist-test-slave-gjw7 krb5kdc[7553](info): commencing operation
krb5kdc: starting...
W20260808 00:41:53.798750 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:53 dist-test-slave-gjw7 krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149713, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:53.919914 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46835
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:42295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:46835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:54.000633 7589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:54.000836 7589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:54.000867 7589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:54.002017 7589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:54.002061 7589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:54.002080 7589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:54.002096 7589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:54.002111 7589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:54.003556 7589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:42295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:46835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:46835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:54.003795 7589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:54.003996 7589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:54.008697 7606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:54.008796 7589 server_base.cc:1034] running on GCE node
W20260808 00:41:54.008711 7600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:54.008711 7603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:54.009172 7589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:54.009507 7589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:54.010665 7589 hybrid_clock.cc:648] HybridClock initialized: now 1786149714010650 us; error 28 us; skew 500 ppm
Aug 08 00:41:54 dist-test-slave-gjw7 krb5kdc[7553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149714, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:54.016975 7589 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:54.017427 7589 webserver.cc:460] Webserver started at http://127.1.208.62:42173/ using document root <none> and password file <none>
I20260808 00:41:54.017622 7589 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:54.017668 7589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:54.017798 7589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:54.018864 7589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "fb8a2267160a40d6a5a4603e4d4ce54c"
format_stamp: "Formatted at 2026-08-08 00:41:54 on dist-test-slave-gjw7"
server_key: "c79f43f43eda9dffa0527e27c0ab31c5"
server_key_iv: "8d2948b3d15d158162b3e3d3cc834858"
server_key_version: "encryptionkey@0"
I20260808 00:41:54.019223 7589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "fb8a2267160a40d6a5a4603e4d4ce54c"
format_stamp: "Formatted at 2026-08-08 00:41:54 on dist-test-slave-gjw7"
server_key: "c79f43f43eda9dffa0527e27c0ab31c5"
server_key_iv: "8d2948b3d15d158162b3e3d3cc834858"
server_key_version: "encryptionkey@0"
I20260808 00:41:54.020469 7589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:54.021111 7619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:54.021257 7589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:54.021364 7589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "fb8a2267160a40d6a5a4603e4d4ce54c"
format_stamp: "Formatted at 2026-08-08 00:41:54 on dist-test-slave-gjw7"
server_key: "c79f43f43eda9dffa0527e27c0ab31c5"
server_key_iv: "8d2948b3d15d158162b3e3d3cc834858"
server_key_version: "encryptionkey@0"
I20260808 00:41:54.021430 7589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:54.024595 7589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:54.026629 7589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:54.026763 7589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:54.030601 7589 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:46835
I20260808 00:41:54.030629 7705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:46835 every 8 connection(s)
I20260808 00:41:54.030926 7589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:54.031802 7706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:54.034111 7706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c: Bootstrap starting.
I20260808 00:41:54.034502 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7589
I20260808 00:41:54.034613 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:54.034709 7706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:54.034794 1856 external_mini_cluster.cc:1428] Setting key edb569de14f0b7d58a78540dea811bef
I20260808 00:41:54.035010 7706 log.cc:824] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:54.035763 7706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c: No bootstrap required, opened a new log
I20260808 00:41:54.037398 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:54.035470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40769 (local address 127.1.208.62:46835)
0808 00:41:54.035650 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:54.035659 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:54.035689 (+ 30us) server_negotiation.cc:407] Connection header received
0808 00:41:54.036208 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:54.036217 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:54.036250 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:54.036407 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:54.036958 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:54.036962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:54.036962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:54.036966 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:54.037010 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:54.037024 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:54.037105 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:54.037134 (+ 29us) server_negotiation.cc:299] Negotiation successful
0808 00:41:54.037172 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260808 00:41:54.038326 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:54.038352 7706 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } }
I20260808 00:41:54.038507 7706 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 00:41:54.038543 7706 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb8a2267160a40d6a5a4603e4d4ce54c, State: Initialized, Role: FOLLOWER
I20260808 00:41:54.038663 7706 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } }
I20260808 00:41:54.038735 7706 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 00:41:54.038757 7706 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 00:41:54.038802 7706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:54.039307 7706 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } }
I20260808 00:41:54.039413 7706 leader_election.cc:302] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb8a2267160a40d6a5a4603e4d4ce54c; no voters:
I20260808 00:41:54.039569 7706 leader_election.cc:288] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 00:41:54.039589 7723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:54.039757 7723 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [term 1 LEADER]: Becoming Leader. State: Replica: fb8a2267160a40d6a5a4603e4d4ce54c, State: Running, Role: LEADER
I20260808 00:41:54.039855 7723 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } }
I20260808 00:41:54.040014 7706 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:54.040198 7723 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb8a2267160a40d6a5a4603e4d4ce54c. Latest consensus state: current_term: 1 leader_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } } }
I20260808 00:41:54.040228 7728 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb8a2267160a40d6a5a4603e4d4ce54c" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 46835 } } }
I20260808 00:41:54.040344 7723 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:54.040393 7728 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:54.040866 7735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:54.041452 7735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:54.042326 7713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:54.039889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60561 (local address 127.1.208.62:46835)
0808 00:41:54.039923 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:54.039925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:54.041395 (+ 1470us) server_negotiation.cc:407] Connection header received
0808 00:41:54.041473 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:54.041474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:54.041479 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:54.041539 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:54.042111 (+ 572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:54.042112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:54.042113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 00:41:54.042119 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:54.042134 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 00:41:54.042138 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:54.042192 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:54.042245 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 00:41:54.042257 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:54.043044 7735 catalog_manager.cc:1347] Generated new cluster ID: 3174723188fa4139967cd31fe8604cd9
I20260808 00:41:54.043093 7735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:54.052587 7735 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:54.052737 7735 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:54.061213 7735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb8a2267160a40d6a5a4603e4d4ce54c: Generated new TSK 0
I20260808 00:41:54.061425 7735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 00:41:54.072846 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7589
2026-08-08T00:41:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1448 ms)
[----------] 18 tests from AuthTokenIssuingTest (26194 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:54 dist-test-slave-gjw7 krb5kdc[7766](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:54 dist-test-slave-gjw7 krb5kdc[7766](info): set up 2 sockets
Aug 08 00:41:54 dist-test-slave-gjw7 krb5kdc[7766](info): commencing operation
krb5kdc: starting...
W20260808 00:41:55.235581 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:55.352754 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45827
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:55.434823 7790 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:55.435029 7790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:55.435045 7790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:55.435063 7790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:55.436187 7790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:55.436232 7790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:55.436246 7790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:55.436256 7790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:55.436270 7790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:55.437728 7790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:45827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:55.437919 7790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:55.438086 7790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:55.442605 7800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:55.442620 7801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:55.442703 7790 server_base.cc:1034] running on GCE node
W20260808 00:41:55.442621 7805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:55.443077 7790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:55.443342 7790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:55.444489 7790 hybrid_clock.cc:648] HybridClock initialized: now 1786149715444465 us; error 42 us; skew 500 ppm
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:55.446869 7790 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:55.447304 7790 webserver.cc:460] Webserver started at http://127.1.208.62:35275/ using document root <none> and password file <none>
I20260808 00:41:55.447496 7790 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:55.447537 7790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:55.447662 7790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:55.448652 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "3252c78b77e34eebac3afed70bd820a6"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "8c08334e083e6bbac55e6b221c0d912c"
server_key_iv: "a6c14a7f0677e47e6c06beb2fa01fc7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.448987 7790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "3252c78b77e34eebac3afed70bd820a6"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "8c08334e083e6bbac55e6b221c0d912c"
server_key_iv: "a6c14a7f0677e47e6c06beb2fa01fc7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.450260 7790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:55.450889 7815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:55.451025 7790 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:55.451105 7790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "3252c78b77e34eebac3afed70bd820a6"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "8c08334e083e6bbac55e6b221c0d912c"
server_key_iv: "a6c14a7f0677e47e6c06beb2fa01fc7d"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.451174 7790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:55.476398 7790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:55.478245 7790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:55.478370 7790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:55.482291 7790 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:45827
I20260808 00:41:55.482342 7922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:45827 every 8 connection(s)
I20260808 00:41:55.482646 7790 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:55.483587 7923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:55.485145 7923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.487360 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7790
I20260808 00:41:55.487474 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
I20260808 00:41:55.487639 1856 external_mini_cluster.cc:1428] Setting key a622196422144190ef7441083627bb06
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:55.494086 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.485834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45827 (local address 127.0.0.1:59886)
0808 00:41:55.485985 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.486018 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.486508 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.486761 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.486763 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.487019 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.487186 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.487190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.488245 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.488247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.488908 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.488911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.489401 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.489402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.489466 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.489928 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.489936 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.491197 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.492731 (+ 1534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.492737 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.492740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.492931 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.493130 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.493131 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.493132 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.493212 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.493551 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.493556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.493617 (+ 61us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.493697 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.493927 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":57,"thread_start_us":76,"threads_started":1}
I20260808 00:41:55.494091 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.485999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59886 (local address 127.1.208.62:45827)
0808 00:41:55.486120 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.486125 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.486156 (+ 31us) server_negotiation.cc:407] Connection header received
0808 00:41:55.486657 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.486659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.486675 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.486748 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.487277 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.488162 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.488989 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.489284 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.489414 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.491271 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.491278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.491278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.491285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.492658 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.492998 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.493001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.493002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.493075 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.493263 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.493264 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.493265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.493490 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.493560 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.493738 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.493854 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260808 00:41:55.494895 7923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.495193 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.495092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34440)
0808 00:41:55.495132 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.495172 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260808 00:41:55.495316 7821 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.208.61@KRBTEST.COM; defaulting to no policy
I20260808 00:41:55.495692 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.495589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34448)
0808 00:41:55.495652 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.495675 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 00:41:55.495781 7923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:43061: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
I20260808 00:41:55.512245 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:43061
--webserver_interface=127.1.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:55.545001 7923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } attempt: 1
I20260808 00:41:55.545594 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.545443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34462)
0808 00:41:55.545508 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.545558 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:41:55.546007 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.545895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34478)
0808 00:41:55.545942 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.545976 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:41:55.546113 7923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:43061: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
W20260808 00:41:55.594071 7930 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:55.594274 7930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:55.594290 7930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:55.594308 7930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:55.595360 7930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:55.595395 7930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:55.595407 7930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:55.595418 7930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:55.595429 7930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:55.596890 7930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:43061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:55.597081 7930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:55.597249 7930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:55.601776 7943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:55.601804 7945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:55.601802 7941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:55.601954 7930 server_base.cc:1034] running on GCE node
I20260808 00:41:55.602195 7930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:55.602464 7930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:55.603603 7930 hybrid_clock.cc:648] HybridClock initialized: now 1786149715603586 us; error 30 us; skew 500 ppm
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:55.605991 7930 init.cc:377] Logged in from keytab as kudu/127.1.208.61@KRBTEST.COM (short username kudu)
I20260808 00:41:55.606406 7930 webserver.cc:460] Webserver started at http://127.1.208.61:36133/ using document root <none> and password file <none>
I20260808 00:41:55.606607 7930 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:55.606652 7930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:55.606783 7930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:55.607789 7930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data/instance:
uuid: "3b32e2f5068b40e788b0eab1ff90d7ab"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "5e9bbb0f6aa7abc1d6e6f7f99e0edb30"
server_key_iv: "fd8b4be95e396b017d0becaacb0504ce"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.608124 7930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal/instance:
uuid: "3b32e2f5068b40e788b0eab1ff90d7ab"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "5e9bbb0f6aa7abc1d6e6f7f99e0edb30"
server_key_iv: "fd8b4be95e396b017d0becaacb0504ce"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.609397 7930 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260808 00:41:55.610082 7961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:55.610222 7930 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:55.610303 7930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "3b32e2f5068b40e788b0eab1ff90d7ab"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "5e9bbb0f6aa7abc1d6e6f7f99e0edb30"
server_key_iv: "fd8b4be95e396b017d0becaacb0504ce"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.610369 7930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:55.614259 7923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } attempt: 2
I20260808 00:41:55.614619 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.614501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34494)
0808 00:41:55.614551 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.614587 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 00:41:55.614874 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.614817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34506)
0808 00:41:55.614838 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.614860 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11}
W20260808 00:41:55.614964 7923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:43061: Network error: Client connection negotiation failed: client connection to 127.1.208.61:43061: connect: Connection refused (error 111)
I20260808 00:41:55.634305 7930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:55.636332 7930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:55.636478 7930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:55.640867 7930 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.61:43061
I20260808 00:41:55.640904 8048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.61:43061 every 8 connection(s)
I20260808 00:41:55.641256 7930 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
I20260808 00:41:55.642273 8049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:55.643837 8049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.647148 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 7930
I20260808 00:41:55.647251 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-1/wal/instance
I20260808 00:41:55.647421 1856 external_mini_cluster.cc:1428] Setting key 74b19125408d81ebfcccddd3b424f11a
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:55.652590 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.644568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.1.208.62:45827)
0808 00:41:55.644687 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.644689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.644862 (+ 173us) server_negotiation.cc:407] Connection header received
0808 00:41:55.645419 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.645420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.645435 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.645481 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.646096 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.646593 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.647632 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.647850 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.647908 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.649983 (+ 2075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.649985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.649986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.649993 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.651253 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.651667 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.651668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.651669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.651746 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.651968 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.651969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.651970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.652115 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.652183 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.652403 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.652494 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 00:41:55.652630 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.644506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45827 (local address 127.0.0.1:59888)
0808 00:41:55.644742 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.644782 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.645307 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.645559 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.645562 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.645810 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.645999 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.646004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.646676 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.646677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.647557 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.647561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.647937 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.647938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.648068 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.648522 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.648530 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.649898 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.651330 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.651338 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.651340 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.651571 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.651821 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.651821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.651822 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.651916 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.652173 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.652176 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.652283 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.652359 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.652472 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260808 00:41:55.653223 8049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.208.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.60@KRBTEST.COM" created.
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:41:55.661844 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.653521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34510)
0808 00:41:55.653582 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.653593 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.653633 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.653892 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.653893 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.654045 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.654116 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.654118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.655045 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.655046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.655537 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.655538 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.656059 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.656061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.656128 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.656442 (+ 314us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.656445 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.659023 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.660884 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.660885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.660886 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.661042 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.661257 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.661258 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.661258 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.661327 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.661621 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.661622 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.661657 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.661705 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.661775 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:41:55.661940 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.653597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34510 (local address 127.1.208.61:43061)
0808 00:41:55.653764 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.653768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.653786 (+ 18us) server_negotiation.cc:407] Connection header received
0808 00:41:55.653803 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.653805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.653816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.653885 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.654243 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.654968 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.655611 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.655986 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.656053 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.659105 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.659114 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.659115 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.659122 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.660803 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.661124 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.661127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.661128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.661194 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.661380 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.661381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.661381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.661562 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.661647 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.661754 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.661835 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":92,"threads_started":1}
I20260808 00:41:55.662760 8049 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.663091 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.662964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35762)
0808 00:41:55.663013 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.663068 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:41:55.663276 7966 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
I20260808 00:41:55.663692 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.663594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35764)
0808 00:41:55.663638 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.663670 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 00:41:55.663817 8049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38255: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
I20260808 00:41:55.673514 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:38255
--webserver_interface=127.1.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:35841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:55.706106 7923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } attempt: 3
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:41:55.712815 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.706535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34518 (local address 127.1.208.61:43061)
0808 00:41:55.706600 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.706602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.706613 (+ 11us) server_negotiation.cc:407] Connection header received
0808 00:41:55.706641 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.706642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.706652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.706699 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.707016 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.707505 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.708189 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.708414 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.708471 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.710583 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.710584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.710585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.710592 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.711781 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.712088 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.712089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.712089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.712140 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.712309 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.712309 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.712310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.712422 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.712479 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.712633 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.712703 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:41:55.712801 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.706458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34518)
0808 00:41:55.706521 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.706532 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.706580 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.706710 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.706711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.706872 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.706938 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.706940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.707577 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.707579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.708110 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.708113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.708478 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.708479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.708551 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.709034 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.709038 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.710503 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.711862 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.711863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.711864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.712028 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.712193 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.712194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.712194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.712257 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.712480 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.712481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.712530 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.712589 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.712693 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:41:55.713021 8049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } attempt: 1
I20260808 00:41:55.713382 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.713264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35772)
0808 00:41:55.713320 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.713356 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 00:41:55.713454 7923 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.713737 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.713636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35788)
0808 00:41:55.713682 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.713710 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:41:55.713804 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.713668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35792)
0808 00:41:55.713732 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.713768 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 00:41:55.713819 8049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38255: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
I20260808 00:41:55.714128 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.714044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35802)
0808 00:41:55.714084 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.714110 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260808 00:41:55.714212 7923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38255: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
I20260808 00:41:55.745361 7923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } attempt: 1
I20260808 00:41:55.745932 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.745775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35804)
0808 00:41:55.745851 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.745895 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 00:41:55.746289 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.746185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35808)
0808 00:41:55.746233 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.746265 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260808 00:41:55.746388 7923 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38255: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
W20260808 00:41:55.756137 8057 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:55.756335 8057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:55.756351 8057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:55.756377 8057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:55.757474 8057 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:55.757510 8057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:55.757524 8057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:55.757541 8057 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:55.757552 8057 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:55.758977 8057 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:35841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:45827,127.1.208.61:43061,127.1.208.60:38255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:38255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:55.759176 8057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:55.759354 8057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:55.763818 8076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:55.763854 8071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:55.763854 8072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:55.764026 8057 server_base.cc:1034] running on GCE node
I20260808 00:41:55.764289 8057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:55.764550 8057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:55.765699 8057 hybrid_clock.cc:648] HybridClock initialized: now 1786149715765686 us; error 35 us; skew 500 ppm
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:55.767951 8057 init.cc:377] Logged in from keytab as kudu/127.1.208.60@KRBTEST.COM (short username kudu)
I20260808 00:41:55.768354 8057 webserver.cc:460] Webserver started at http://127.1.208.60:45103/ using document root <none> and password file <none>
I20260808 00:41:55.768544 8057 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:55.768590 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:55.768712 8057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:55.769737 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data/instance:
uuid: "58c456af70464e0dad5c04c3c1021254"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "25ecf52a9a40199603e79d0451e04350"
server_key_iv: "65f30aa572629d7d81d02f886c15cfb3"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.770067 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal/instance:
uuid: "58c456af70464e0dad5c04c3c1021254"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "25ecf52a9a40199603e79d0451e04350"
server_key_iv: "65f30aa572629d7d81d02f886c15cfb3"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.771270 8057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 00:41:55.771915 8089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:55.772086 8057 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:55.772189 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "58c456af70464e0dad5c04c3c1021254"
format_stamp: "Formatted at 2026-08-08 00:41:55 on dist-test-slave-gjw7"
server_key: "25ecf52a9a40199603e79d0451e04350"
server_key_iv: "65f30aa572629d7d81d02f886c15cfb3"
server_key_version: "encryptionkey@0"
I20260808 00:41:55.772251 8057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:55.778676 8057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:55.780375 8057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:55.780498 8057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:55.781957 8049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } attempt: 2
I20260808 00:41:55.782373 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.782210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35824)
0808 00:41:55.782279 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.782331 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 00:41:55.782761 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.782655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35830)
0808 00:41:55.782698 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.782736 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 00:41:55.782846 8049 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:38255: Network error: Client connection negotiation failed: client connection to 127.1.208.60:38255: connect: Connection refused (error 111)
I20260808 00:41:55.784730 8057 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.60:38255
I20260808 00:41:55.784765 8181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.60:38255 every 8 connection(s)
I20260808 00:41:55.785076 8057 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
I20260808 00:41:55.786057 8182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:55.787582 8182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.788600 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8057
I20260808 00:41:55.788709 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786149440113891-1856-0/minicluster-data/master-2/wal/instance
I20260808 00:41:55.788870 1856 external_mini_cluster.cc:1428] Setting key 0fc6df00b06a33bc29cdb72e7bca697a
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:55.796526 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.788229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.1.208.62:45827)
0808 00:41:55.788274 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.788276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.788460 (+ 184us) server_negotiation.cc:407] Connection header received
0808 00:41:55.789001 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.789003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.789014 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.789068 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.789761 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.790303 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.791217 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.791403 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.791457 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.793693 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.793694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.793695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.793701 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.794998 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.795465 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.795466 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.795467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.795527 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.795793 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.795793 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.795794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.795945 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.796023 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.796352 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.796429 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:41:55.796607 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.788172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:45827 (local address 127.0.0.1:59890)
0808 00:41:55.788353 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.788388 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.788911 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.789144 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.789148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.789490 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.789675 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.789679 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.790378 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.790379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.791140 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.791142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.791490 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.791491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.791605 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.792053 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.792061 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.793618 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.795077 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.795086 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.795088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.795402 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.795594 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.795594 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.795595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.795732 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.796067 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.796074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.796198 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.796299 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.796438 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260808 00:41:55.797200 8182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.797927 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.789215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.1.208.60:38255)
0808 00:41:55.789801 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.789805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.789821 (+ 16us) server_negotiation.cc:407] Connection header received
0808 00:41:55.789839 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.789841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.789869 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.789942 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.790548 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.791298 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.792100 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.792396 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.792452 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.794813 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.794821 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.794822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.794841 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.796613 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.796995 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.796999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.796999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.797066 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.797258 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.797258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.797259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.797447 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.797585 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.797794 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.797858 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":100,"threads_started":1}
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:55.803746 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.797460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:43061 (local address 127.0.0.1:34530)
0808 00:41:55.797507 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.797517 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.797551 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.797650 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.797650 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.797811 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.797872 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.797874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.798627 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.798628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.799365 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.799368 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.799713 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.799714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.799792 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.800121 (+ 329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.800126 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.801200 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.802753 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.802754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.802754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.802956 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.803164 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.803165 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.803166 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.803222 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.803458 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.803460 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.803504 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.803563 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.803650 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 00:41:55.803757 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.797520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.1.208.61:43061)
0808 00:41:55.797560 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.797562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.797570 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:55.797596 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.797597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.797605 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.797644 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.797950 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.798551 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.799436 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.799654 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.799707 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.801412 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.801413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.801414 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.801422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.802679 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.803044 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.803045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.803045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.803104 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.803280 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.803281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.803281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.803383 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.803442 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.803594 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.803672 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:55.804167 8182 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:55.805770 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.798640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.1.208.62:45827)
0808 00:41:55.798673 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.798674 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.798681 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:55.798733 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.798733 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.798740 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.798775 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.799285 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.799691 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.800311 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.800502 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.800552 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.803127 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.803128 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.803129 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.803135 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.804613 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.804937 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.804938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.804938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.805001 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.805215 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.805217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.805217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.805387 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.805531 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.805596 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.805681 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:55.810369 8184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.804392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35848)
0808 00:41:55.804541 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.804565 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.804630 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.804749 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.804750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.804848 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.804874 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.804875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.805617 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.805618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.806164 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.806166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.806593 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.806594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.806642 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.806978 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.806981 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.808142 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.809543 (+ 1401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.809544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.809544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.809745 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.809910 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.809910 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.809911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.809970 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.810149 (+ 179us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.810150 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.810192 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.810253 (+ 61us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.810315 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260808 00:41:55.810477 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.804475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35848 (local address 127.1.208.60:38255)
0808 00:41:55.804508 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.804510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.804615 (+ 105us) server_negotiation.cc:407] Connection header received
0808 00:41:55.804684 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.804685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.804695 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.804744 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.804934 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.805549 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.806296 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.806528 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.806585 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.808213 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.808214 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.808214 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.808220 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.809474 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.809813 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.809814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.809814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.809857 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.810019 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.810019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.810019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.810099 (+ 80us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.810141 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.810290 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.810391 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":19}
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:41:55.811643 8182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: Bootstrap starting.
I20260808 00:41:55.812433 8182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:55.812700 8182 log.cc:824] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:55.813520 8182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: No bootstrap required, opened a new log
I20260808 00:41:55.815225 8182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.815387 8182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58c456af70464e0dad5c04c3c1021254, State: Initialized, Role: FOLLOWER
I20260808 00:41:55.815483 8182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.815580 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.806407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34536 (local address 127.1.208.61:43061)
0808 00:41:55.806440 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.806441 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.806449 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:55.806496 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.806496 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.806503 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.806540 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.807087 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.807640 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.808461 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.808734 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.808808 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.813264 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.813265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.813265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.813271 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.814568 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.814930 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.814930 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.814931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.814973 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.815133 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.815133 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.815133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.815219 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.815331 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.815419 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.815490 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:41:55.815845 8182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:55.815799 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.815908 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.817092 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 00:41:55.817996 8211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:55.818090 8211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:55.818302 7868 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3252c78b77e34eebac3afed70bd820a6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59902
I20260808 00:41:55.817986 8057 master_runner.cc:186] Error getting master registration for 127.1.208.62:45827: OK, instance_id { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" instance_seqno: 1786149715481191 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:41:55.820900 8057 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:41:55.821511 7923 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } attempt: 2
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:55.827870 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.821836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.1.208.60:38255)
0808 00:41:55.821888 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.821890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.821896 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:41:55.821916 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.821916 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.821925 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.821973 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.822279 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.822746 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.823387 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.823592 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.823645 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.825598 (+ 1953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.825599 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.825599 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.825607 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.826858 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.827156 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.827157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.827157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.827211 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.827379 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.827380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.827380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.827512 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.827568 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.827705 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.827774 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:41:55.827870 7925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.821754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35864)
0808 00:41:55.821802 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.821812 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.821848 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.821984 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.821985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.822157 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.822201 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.822203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.822812 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.822813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.823309 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.823310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.823651 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.823652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.823708 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.824067 (+ 359us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.824072 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.825521 (+ 1449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.826932 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.826933 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.826933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.827095 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.827262 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.827263 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.827263 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.827330 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.827569 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.827570 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.827617 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.827675 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.827774 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 00:41:55.829178 7923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: Bootstrap starting.
I20260808 00:41:55.829816 7923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:55.830102 7923 log.cc:824] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:55.830837 7923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: No bootstrap required, opened a new log
I20260808 00:41:55.831866 7923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.831976 7923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3252c78b77e34eebac3afed70bd820a6, State: Initialized, Role: FOLLOWER
I20260808 00:41:55.832036 7923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.832343 7923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:55.832293 8215 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.832384 7868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:41:55.832399 8215 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.832471 7868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:55.833243 7868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.833623 7868 leader_election.cc:288] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [CANDIDATE]: Term 1 election: Requested vote from peers 3b32e2f5068b40e788b0eab1ff90d7ab (127.1.208.61:43061), 58c456af70464e0dad5c04c3c1021254 (127.1.208.60:38255)
I20260808 00:41:55.834163 8134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3252c78b77e34eebac3afed70bd820a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "58c456af70464e0dad5c04c3c1021254"
I20260808 00:41:55.834312 8134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [term 0 FOLLOWER]: Advancing to term 1
W20260808 00:41:55.834344 8239 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:55.834386 8239 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:55.835258 8134 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3252c78b77e34eebac3afed70bd820a6 in term 1.
I20260808 00:41:55.835530 7816 leader_election.cc:302] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3252c78b77e34eebac3afed70bd820a6, 58c456af70464e0dad5c04c3c1021254; no voters:
I20260808 00:41:55.835704 8215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:55.835834 8215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [term 1 LEADER]: Becoming Leader. State: Replica: 3252c78b77e34eebac3afed70bd820a6, State: Running, Role: LEADER
I20260808 00:41:55.835919 8215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.839110 8242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3252c78b77e34eebac3afed70bd820a6. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.839303 8242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:55.837831 7790 master_runner.cc:186] Error getting master registration for 127.1.208.61:43061: OK, instance_id { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" instance_seqno: 1786149715639659 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:41:55.840184 7790 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:41:55.841780 8256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:55.842356 8256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:55.851171 8054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.838527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34538 (local address 127.1.208.61:43061)
0808 00:41:55.838570 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.838572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.843255 (+ 4683us) server_negotiation.cc:407] Connection header received
0808 00:41:55.843352 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.843353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.843364 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.843391 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.844016 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.844759 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.845801 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.846113 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.846152 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.847600 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.847601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.847602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.847609 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.849406 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.849853 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.849854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.849855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.849911 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.850126 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.850127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.850128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.850269 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.850777 (+ 508us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.850993 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.851071 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:55.852058 7926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.837811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59908 (local address 127.1.208.62:45827)
0808 00:41:55.837861 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.837864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.841492 (+ 3628us) server_negotiation.cc:407] Connection header received
0808 00:41:55.841558 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.841559 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.841581 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.841629 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.842777 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.843509 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.845086 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.845434 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.845479 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.847720 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.847722 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.847722 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.847729 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.849550 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.850119 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.850120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.850120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.850195 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.850479 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.850480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.850480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.850612 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.851858 (+ 1246us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.851899 (+ 41us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.851976 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 00:41:55.853477 8134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [term 1 FOLLOWER]: Refusing update from remote peer 3252c78b77e34eebac3afed70bd820a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:55.853817 8215 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:55.855412 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3252c78b77e34eebac3afed70bd820a6. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.855504 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.856024 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.838076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.1.208.60:38255)
0808 00:41:55.838118 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.838120 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.841622 (+ 3502us) server_negotiation.cc:407] Connection header received
0808 00:41:55.841681 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.841682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.841692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.841742 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.849871 (+ 8129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.850530 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.851425 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.851695 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.851759 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.853083 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.853084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.853084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.853091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.854699 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.855064 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.855065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.855066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.855118 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.855312 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.855313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.855313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.855432 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.855508 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.855847 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.855917 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260808 00:41:55.859663 8242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.859748 8242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:55.860059 8256 catalog_manager.cc:1347] Generated new cluster ID: 698ea6de94ae4c92a35a727c7654a3fd
I20260808 00:41:55.860106 8256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:55.860275 8194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.860360 8194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.860737 8262 mvcc.cc:205] Tried to move back new op lower bound from 7316069236132483072 to 7316069236064636928. Current Snapshot: MvccSnapshot[applied={T|T < 7316069236132483072}]
I20260808 00:41:55.873962 8049 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } attempt: 3
Aug 08 00:41:55 dist-test-slave-gjw7 krb5kdc[7766](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149715, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:55.884647 8051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.874305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:38255 (local address 127.0.0.1:35884)
0808 00:41:55.874404 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:55.874415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:55.874455 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:55.874717 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:55.874718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:55.874904 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:55.874951 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.874953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.875948 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.875949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.876774 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:55.876776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.878133 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.878134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:55.878218 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.879200 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:55.879206 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:55.880886 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:55.883427 (+ 2541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.883428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.883429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.883654 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.883905 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:55.883906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:55.883906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:55.883983 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:55.884303 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:55.884304 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:55.884376 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 00:41:55.884435 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 00:41:55.884537 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 00:41:55.885243 8190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:55.874545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.1.208.60:38255)
0808 00:41:55.874585 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.874586 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.874593 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:55.874613 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.874614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.874623 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.874665 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.875027 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.875804 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.876864 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:55.877177 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:55.877230 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:55.881001 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:55.881003 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:55.881003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:55.881028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:55.883313 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.883743 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.883744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.883745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.883822 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:55.884056 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:55.884057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:55.884057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:55.884205 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:55.884260 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:55.885006 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 00:41:55.885086 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:41:55.887184 8256 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:55.888717 8256 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:55.889108 8049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: Bootstrap starting.
I20260808 00:41:55.889882 8049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:55.890218 8049 log.cc:824] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:55.890997 8049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: No bootstrap required, opened a new log
I20260808 00:41:55.892961 8049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.893101 8049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b32e2f5068b40e788b0eab1ff90d7ab, State: Initialized, Role: FOLLOWER
I20260808 00:41:55.893175 8049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } }
I20260808 00:41:55.893539 8049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:55.893608 8012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3252c78b77e34eebac3afed70bd820a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab"
I20260808 00:41:55.893721 8012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:55.894569 8012 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3252c78b77e34eebac3afed70bd820a6 in term 1.
I20260808 00:41:55.894784 8011 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [term 1 FOLLOWER]: Refusing update from remote peer 3252c78b77e34eebac3afed70bd820a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:55.895196 8242 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:55.897411 8269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.897567 8269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.897665 8269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3252c78b77e34eebac3afed70bd820a6. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.897759 8269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.897822 8269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.897903 8269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:55.901899 8286 mvcc.cc:205] Tried to move back new op lower bound from 7316069236237946880 to 7316069236064636928. Current Snapshot: MvccSnapshot[applied={T|T < 7316069236132483072}]
W20260808 00:41:55.903086 8294 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:55.903434 8294 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:55.904452 8242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3252c78b77e34eebac3afed70bd820a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3252c78b77e34eebac3afed70bd820a6" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 45827 } } peers { permanent_uuid: "3b32e2f5068b40e788b0eab1ff90d7ab" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 43061 } } peers { permanent_uuid: "58c456af70464e0dad5c04c3c1021254" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 38255 } } }
I20260808 00:41:55.904543 8242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6 [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:55.905402 8256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3252c78b77e34eebac3afed70bd820a6: Generated new TSK 0
I20260808 00:41:55.905579 8256 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:41:55.930357 7926 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.928863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.1.208.62:45827)
0808 00:41:55.928909 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.928911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.929664 (+ 753us) server_negotiation.cc:407] Connection header received
0808 00:41:55.929782 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.929783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.929795 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.929825 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.930271 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59912: BlockingRecv error: recv got EOF from 127.0.0.1:59912 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260808 00:41:55.931268 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.929333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.1.208.61:43061)
0808 00:41:55.929379 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.929381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.930760 (+ 1379us) server_negotiation.cc:407] Connection header received
0808 00:41:55.930855 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.930856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.930866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.930893 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.931206 (+ 313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34546: BlockingRecv error: recv got EOF from 127.0.0.1:34546 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:41:55.932479 8190 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.931743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35888 (local address 127.1.208.60:38255)
0808 00:41:55.931784 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.931786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.931912 (+ 126us) server_negotiation.cc:407] Connection header received
0808 00:41:55.932032 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.932033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.932057 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.932084 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.932417 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35888: BlockingRecv error: recv got EOF from 127.0.0.1:35888 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260808 00:41:55.934731 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.933976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34554 (local address 127.1.208.61:43061)
0808 00:41:55.934009 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.934011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.934234 (+ 223us) server_negotiation.cc:407] Connection header received
0808 00:41:55.934347 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.934348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.934366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.934394 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.934686 (+ 292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34554: BlockingRecv error: recv got EOF from 127.0.0.1:34554 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 00:41:55.937783 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.936977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.1.208.61:43061)
0808 00:41:55.937009 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.937011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.937251 (+ 240us) server_negotiation.cc:407] Connection header received
0808 00:41:55.937390 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.937391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.937402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.937429 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.937741 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34556: BlockingRecv error: recv got EOF from 127.0.0.1:34556 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 00:41:55.941848 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.941052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.1.208.61:43061)
0808 00:41:55.941084 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.941085 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.941356 (+ 271us) server_negotiation.cc:407] Connection header received
0808 00:41:55.941456 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.941457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.941467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.941495 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.941805 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34564: BlockingRecv error: recv got EOF from 127.0.0.1:34564 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 00:41:55.947924 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.947138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34566 (local address 127.1.208.61:43061)
0808 00:41:55.947171 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.947173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.947427 (+ 254us) server_negotiation.cc:407] Connection header received
0808 00:41:55.947522 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.947523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.947533 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.947562 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.947883 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34566: BlockingRecv error: recv got EOF from 127.0.0.1:34566 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:41:55.958266 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.957195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34578 (local address 127.1.208.61:43061)
0808 00:41:55.957256 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.957257 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.957731 (+ 474us) server_negotiation.cc:407] Connection header received
0808 00:41:55.957831 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.957832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.957843 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.957871 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.958204 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34578: BlockingRecv error: recv got EOF from 127.0.0.1:34578 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:41:55.976775 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:55.975781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34580 (local address 127.1.208.61:43061)
0808 00:41:55.975840 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:55.975842 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:55.976178 (+ 336us) server_negotiation.cc:407] Connection header received
0808 00:41:55.976280 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:55.976281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:55.976293 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:55.976321 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:55.976716 (+ 395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34580: BlockingRecv error: recv got EOF from 127.0.0.1:34580 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 00:41:56.010896 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:56.010165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.1.208.61:43061)
0808 00:41:56.010218 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.010220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.010348 (+ 128us) server_negotiation.cc:407] Connection header received
0808 00:41:56.010424 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.010425 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.010436 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.010479 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:56.010817 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34586: BlockingRecv error: recv got EOF from 127.0.0.1:34586 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:41:56.076656 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:56.075915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.1.208.61:43061)
0808 00:41:56.075984 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.075987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.076109 (+ 122us) server_negotiation.cc:407] Connection header received
0808 00:41:56.076187 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.076188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.076200 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.076242 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:56.076589 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34598: BlockingRecv error: recv got EOF from 127.0.0.1:34598 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260808 00:41:56.206444 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:56.205754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34610 (local address 127.1.208.61:43061)
0808 00:41:56.205828 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.205830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.205897 (+ 67us) server_negotiation.cc:407] Connection header received
0808 00:41:56.205971 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.205972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.205984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.206034 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:56.206394 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34610: BlockingRecv error: recv got EOF from 127.0.0.1:34610 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 00:41:56.464291 8054 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:56.463741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34620 (local address 127.1.208.61:43061)
0808 00:41:56.463804 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.463806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.463841 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:41:56.463896 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.463897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.463907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.463953 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:56.464241 (+ 288us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34620: BlockingRecv error: recv got EOF from 127.0.0.1:34620 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:41:56.818552 8211 catalog_manager.cc:1259] Loaded cluster ID: 698ea6de94ae4c92a35a727c7654a3fd
I20260808 00:41:56.818631 8211 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: loading cluster ID for follower catalog manager: success
I20260808 00:41:56.819432 8211 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: acquiring CA information for follower catalog manager: success
I20260808 00:41:56.819736 8211 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 58c456af70464e0dad5c04c3c1021254: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:41:56.903965 8294 catalog_manager.cc:1259] Loaded cluster ID: 698ea6de94ae4c92a35a727c7654a3fd
I20260808 00:41:56.904052 8294 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: loading cluster ID for follower catalog manager: success
I20260808 00:41:56.904832 8294 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: acquiring CA information for follower catalog manager: success
I20260808 00:41:56.905097 8294 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3b32e2f5068b40e788b0eab1ff90d7ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:41:56.980151 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:56.977625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34626 (local address 127.1.208.61:43061)
0808 00:41:56.977735 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.977737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.977766 (+ 29us) server_negotiation.cc:407] Connection header received
0808 00:41:56.977831 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.977832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.977886 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.977968 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:56.978096 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.978623 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.979440 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.979641 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.979696 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:56.979808 (+ 112us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:56.979920 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:56.979974 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:56.980062 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:41:56.980079 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":59,"threads_started":1}
I20260808 00:41:56.984407 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:56.981917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35894 (local address 127.1.208.60:38255)
0808 00:41:56.982023 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.982025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.982034 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:56.982105 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.982106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.982154 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.982218 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:56.982321 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.982799 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.983576 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.983812 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.983866 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:56.984022 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:56.984128 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:56.984192 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:56.984296 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 00:41:56.984320 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
I20260808 00:41:56.988651 8413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:56.986244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.1.208.60:38255)
0808 00:41:56.986278 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.986279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.986287 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:56.986390 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.986391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.986402 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.986450 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:56.986565 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.987258 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988017 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.988210 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988262 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:56.988423 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:56.988456 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:56.988505 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:56.988560 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 00:41:56.988584 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:56.988982 8406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:56.986117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.1.208.61:43061)
0808 00:41:56.986156 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.986158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.986193 (+ 35us) server_negotiation.cc:407] Connection header received
0808 00:41:56.986316 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.986317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.986330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.986358 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:56.986520 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.987260 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988251 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.988523 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988576 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:56.988744 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:56.988778 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:56.988803 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:56.988891 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 00:41:56.988914 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:56.989234 8428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:56.986123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59918 (local address 127.1.208.62:45827)
0808 00:41:56.986342 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:56.986344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:56.986351 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:56.986396 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:56.986397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:56.986449 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:56.986523 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:56.986650 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.987151 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988286 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:56.988538 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:56.988591 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:56.988840 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:56.988943 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:56.989002 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:56.989125 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 00:41:56.989148 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":50,"threads_started":1}
I20260808 00:41:56.990684 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7790
I20260808 00:41:57.001209 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 7930
I20260808 00:41:57.010376 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8057
2026-08-08T00:41:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2944 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 00:41:57 dist-test-slave-gjw7 krb5kdc[8435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 00:41:57 dist-test-slave-gjw7 krb5kdc[8435](info): set up 2 sockets
Aug 08 00:41:57 dist-test-slave-gjw7 krb5kdc[8435](info): commencing operation
krb5kdc: starting...
W20260808 00:41:58.200342 1856 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/test-user.keytab.
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T00:41:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T00:41:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.208.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
Entry for principal kudu/127.1.208.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab.
I20260808 00:41:58.318236 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:44143
--webserver_interface=127.1.208.62
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 00:41:58.402422 8457 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:58.402627 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:58.402643 8457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:58.402662 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:58.403822 8457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:58.403867 8457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:58.403878 8457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:58.403889 8457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:58.403903 8457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:58.405436 8457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.62.keytab
--principal=kudu/127.1.208.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.62:44143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.208.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:58.405669 8457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:58.405884 8457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:58.410466 8466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:58.410588 8471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:58.410691 8468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:58.410781 8457 server_base.cc:1034] running on GCE node
I20260808 00:41:58.411039 8457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:58.411310 8457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:58.412470 8457 hybrid_clock.cc:648] HybridClock initialized: now 1786149718412456 us; error 36 us; skew 500 ppm
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:58.414943 8457 init.cc:377] Logged in from keytab as kudu/127.1.208.62@KRBTEST.COM (short username kudu)
I20260808 00:41:58.415370 8457 webserver.cc:460] Webserver started at http://127.1.208.62:33085/ using document root <none> and password file <none>
I20260808 00:41:58.415566 8457 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:58.415606 8457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:58.415690 8457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:58.416721 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data/instance:
uuid: "ffda26169fca4d479df6337f4ed25c8a"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "6cd56730be0f25db8717e4834a00000c"
server_key_iv: "f627b4c986e86818fc637de620913df1"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.417032 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance:
uuid: "ffda26169fca4d479df6337f4ed25c8a"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "6cd56730be0f25db8717e4834a00000c"
server_key_iv: "f627b4c986e86818fc637de620913df1"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.418324 8457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260808 00:41:58.418988 8483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:58.419142 8457 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:58.419219 8457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal
uuid: "ffda26169fca4d479df6337f4ed25c8a"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "6cd56730be0f25db8717e4834a00000c"
server_key_iv: "f627b4c986e86818fc637de620913df1"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.419324 8457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:58.429174 8457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:58.430918 8457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:58.431035 8457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:58.434960 8457 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.62:44143
I20260808 00:41:58.435009 8570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.62:44143 every 8 connection(s)
I20260808 00:41:58.435344 8457 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/data/info.pb
I20260808 00:41:58.436326 8571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:58.437958 8571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.443629 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8457
I20260808 00:41:58.443760 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-0/wal/instance
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:58.443979 1856 external_mini_cluster.cc:1428] Setting key 46ff4d1a94250ff1ad3dcea9602a2a26
I20260808 00:41:58.447325 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.438658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.1.208.62:44143)
0808 00:41:58.438847 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.438856 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.438947 (+ 91us) server_negotiation.cc:407] Connection header received
0808 00:41:58.439518 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.439520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.439536 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.439611 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.440130 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.440953 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.441819 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.442110 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.442213 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.444334 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.444343 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.444343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.444351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.445898 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.446321 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.446323 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.446323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.446400 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.446610 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.446611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.446611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.446798 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.446862 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.447071 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.447161 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":73,"threads_started":1}
I20260808 00:41:58.447392 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.438626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44143 (local address 127.0.0.1:60574)
0808 00:41:58.438804 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.438847 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.439369 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.439624 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.439627 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.439894 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.440040 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.440045 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.441039 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.441041 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.441741 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.441744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.442198 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.442200 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.442263 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.442699 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.442708 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.444227 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.445975 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.445983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.445986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.446233 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.446455 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.446456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.446456 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.446557 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.446862 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.446868 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.446944 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.447038 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.447244 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":58,"thread_start_us":91,"threads_started":1}
I20260808 00:41:58.448058 8571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.448350 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.448261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42878)
0808 00:41:58.448293 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.448333 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260808 00:41:58.448477 8485 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
I20260808 00:41:58.448861 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.448779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42880)
0808 00:41:58.448822 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.448844 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260808 00:41:58.448947 8571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:34983: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.208.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
Entry for principal kudu/127.1.208.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab.
I20260808 00:41:58.467851 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:34983
--webserver_interface=127.1.208.61
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:58.498162 8571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } attempt: 1
I20260808 00:41:58.498694 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.498546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42884)
0808 00:41:58.498610 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.498649 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 00:41:58.499075 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.498978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42894)
0808 00:41:58.499023 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.499052 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:41:58.499224 8571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:34983: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
W20260808 00:41:58.550734 8584 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:58.550947 8584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:58.550971 8584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:58.550997 8584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:58.552111 8584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:58.552167 8584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:58.552187 8584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:58.552203 8584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:58.552219 8584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:58.553741 8584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.61.keytab
--principal=kudu/127.1.208.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.61:34983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.208.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:58.553980 8584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:58.554180 8584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:58.558779 8591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:58.558759 8595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:58.558898 8584 server_base.cc:1034] running on GCE node
W20260808 00:41:58.558895 8590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:58.559213 8584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:58.559500 8584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:58.560647 8584 hybrid_clock.cc:648] HybridClock initialized: now 1786149718560633 us; error 26 us; skew 500 ppm
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:58.563091 8584 init.cc:377] Logged in from keytab as kudu/127.1.208.61@KRBTEST.COM (short username kudu)
I20260808 00:41:58.563485 8584 webserver.cc:460] Webserver started at http://127.1.208.61:40051/ using document root <none> and password file <none>
I20260808 00:41:58.563679 8584 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:58.563725 8584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:58.563859 8584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:58.564872 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data/instance:
uuid: "3bc4e16f761742f59b4300279bfcad69"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "175e1c58f2a657d00bcd0bb253c16981"
server_key_iv: "4bd44f5d13639ec0dcb4c8fe64c329fb"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.565201 8584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal/instance:
uuid: "3bc4e16f761742f59b4300279bfcad69"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "175e1c58f2a657d00bcd0bb253c16981"
server_key_iv: "4bd44f5d13639ec0dcb4c8fe64c329fb"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.566488 8584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260808 00:41:58.567133 8607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:58.567289 8584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260808 00:41:58.567373 8584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal
uuid: "3bc4e16f761742f59b4300279bfcad69"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "175e1c58f2a657d00bcd0bb253c16981"
server_key_iv: "4bd44f5d13639ec0dcb4c8fe64c329fb"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.567377 8571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } attempt: 2
I20260808 00:41:58.567440 8584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:58.567761 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.567631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42902)
0808 00:41:58.567681 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.567731 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 00:41:58.568063 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.567976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42904)
0808 00:41:58.568015 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.568038 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 00:41:58.568149 8571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.61:34983: Network error: Client connection negotiation failed: client connection to 127.1.208.61:34983: connect: Connection refused (error 111)
I20260808 00:41:58.587625 8584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:58.589668 8584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:58.589805 8584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:58.593660 8584 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.61:34983
I20260808 00:41:58.593693 8692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.61:34983 every 8 connection(s)
I20260808 00:41:58.594033 8584 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/data/info.pb
I20260808 00:41:58.594998 8693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:58.596526 8693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:58.603189 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8584
I20260808 00:41:58.603283 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-1/wal/instance
I20260808 00:41:58.603487 1856 external_mini_cluster.cc:1428] Setting key 3d743672d88c7dfa21e7219879eb43ab
I20260808 00:41:58.604882 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.597311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60586 (local address 127.1.208.62:44143)
0808 00:41:58.597371 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.597373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.597486 (+ 113us) server_negotiation.cc:407] Connection header received
0808 00:41:58.597989 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.597990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.598001 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.598041 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.598625 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.599096 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.600055 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.600238 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.600283 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.602418 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.602419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.602420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.602427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.603651 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.604017 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.604018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.604018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.604078 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.604286 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.604287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.604288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.604406 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.604462 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.604711 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.604791 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 00:41:58.605144 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.597147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44143 (local address 127.0.0.1:60586)
0808 00:41:58.597364 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.597404 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.597904 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.598096 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.598099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.598356 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.598557 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.598561 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.599174 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.599176 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.599983 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.599986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.600346 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.600348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.600457 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.600961 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.600969 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.602327 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.603716 (+ 1389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.603723 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.603726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.603948 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.604129 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.604130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.604130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.604231 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.604459 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.604466 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.604597 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.604680 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.604822 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":77,"threads_started":1}
I20260808 00:41:58.605732 8693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.208.60@KRBTEST.COM; defaulting to no policy
I20260808 00:41:58.612531 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.605961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42912)
0808 00:41:58.606003 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.606014 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.606056 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.606281 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.606282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.606412 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.606466 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.606467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.607367 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.607369 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.608093 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.608097 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.608532 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.608532 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.608614 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.608938 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.608943 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.610124 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.611612 (+ 1488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.611613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.611614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.611761 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.611931 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.611932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.611932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.611985 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.612229 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.612230 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.612288 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.612374 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.612470 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 00:41:58.612639 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.606021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.1.208.61:34983)
0808 00:41:58.606158 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.606162 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.606176 (+ 14us) server_negotiation.cc:407] Connection header received
0808 00:41:58.606193 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.606195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.606205 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.606261 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.606527 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.607283 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.608170 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.608467 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.608516 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.610196 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.610204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.610204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.610210 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.611541 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.611828 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.611831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.611832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.611879 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.612034 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.612035 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.612035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.612158 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.612233 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.612400 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.612528 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260808 00:41:58.613125 8693 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.613448 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.613348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57876)
0808 00:41:58.613388 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.613431 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 00:41:58.613559 8609 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
I20260808 00:41:58.613967 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.613858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57890)
0808 00:41:58.613919 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.613948 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 00:41:58.614555 8693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:39041: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.208.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
Entry for principal kudu/127.1.208.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab.
I20260808 00:41:58.628034 1856 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasknSGkLY/build/release/bin/kudu
/tmp/dist-test-tasknSGkLY/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:39041
--webserver_interface=127.1.208.60
--webserver_port=0
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.208.20:37117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 00:41:58.659307 8571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } attempt: 3
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:41:58.663766 8693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } attempt: 1
I20260808 00:41:58.664127 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.664012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57898)
0808 00:41:58.664066 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.664090 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:41:58.664450 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.664375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57900)
0808 00:41:58.664411 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.664430 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260808 00:41:58.664539 8693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:39041: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
I20260808 00:41:58.666076 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.659767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.1.208.61:34983)
0808 00:41:58.659829 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.659831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.659841 (+ 10us) server_negotiation.cc:407] Connection header received
0808 00:41:58.659873 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.659874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.659886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.659932 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.660243 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.660732 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.661465 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.661684 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.661742 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.663617 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.663619 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.663619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.663626 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.665057 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.665458 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.665459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.665459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.665510 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.665683 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.665683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.665683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.665777 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.665819 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.665965 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.666023 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 00:41:58.666194 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.659685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42918)
0808 00:41:58.659748 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.659758 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.659805 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.659947 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.659948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.660106 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.660173 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.660175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.660805 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.660807 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.661387 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.661390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.661747 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.661749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.661830 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.662277 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.662282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.663524 (+ 1242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.665134 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.665136 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.665136 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.665395 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.665563 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.665564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.665564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.665630 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.665824 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.665825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.665879 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.665950 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.666044 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260808 00:41:58.666550 8571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.666859 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.666763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57908)
0808 00:41:58.666806 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.666838 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260808 00:41:58.667124 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.667050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57918)
0808 00:41:58.667086 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.667108 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 00:41:58.667205 8571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:39041: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
I20260808 00:41:58.698355 8571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } attempt: 1
I20260808 00:41:58.698814 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.698694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57920)
0808 00:41:58.698752 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.698787 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 00:41:58.699164 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.699085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57934)
0808 00:41:58.699123 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.699147 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260808 00:41:58.699262 8571 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:39041: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
W20260808 00:41:58.710474 8706 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 00:41:58.710670 8706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 00:41:58.710686 8706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 00:41:58.710711 8706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 00:41:58.711784 8706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 00:41:58.711817 8706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 00:41:58.711829 8706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 00:41:58.711840 8706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 00:41:58.711854 8706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 00:41:58.713325 8706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.208.20:37117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.208.62:44143,127.1.208.61:34983,127.1.208.60:39041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/krb5kdc/kudu_127.1.208.60.keytab
--principal=kudu/127.1.208.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.208.60:39041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.208.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 07 Aug 2026 23:15:27 UTC on ccycloud.quasar-ffioya.root.comops.site
build id 4573507
I20260808 00:41:58.713517 8706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 00:41:58.713685 8706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 00:41:58.718096 8722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:58.718119 8719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 00:41:58.718190 8717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 00:41:58.718308 8706 server_base.cc:1034] running on GCE node
I20260808 00:41:58.718567 8706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 00:41:58.718866 8706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 00:41:58.720028 8706 hybrid_clock.cc:648] HybridClock initialized: now 1786149718720008 us; error 30 us; skew 500 ppm
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 00:41:58.722505 8706 init.cc:377] Logged in from keytab as kudu/127.1.208.60@KRBTEST.COM (short username kudu)
I20260808 00:41:58.722949 8706 webserver.cc:460] Webserver started at http://127.1.208.60:34013/ using document root <none> and password file <none>
I20260808 00:41:58.723143 8706 fs_manager.cc:359] Metadata directory not provided
I20260808 00:41:58.723189 8706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 00:41:58.723313 8706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 00:41:58.724310 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data/instance:
uuid: "d3001202d8964b52a96b5337df3290d3"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "a7be59076509d4db148ed15bc16be8ab"
server_key_iv: "ed63ed55c097092f7e398e9f8bda9497"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.724646 8706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal/instance:
uuid: "d3001202d8964b52a96b5337df3290d3"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "a7be59076509d4db148ed15bc16be8ab"
server_key_iv: "ed63ed55c097092f7e398e9f8bda9497"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.725935 8706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260808 00:41:58.726572 8734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 00:41:58.726706 8706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260808 00:41:58.726802 8706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data,/tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal
uuid: "d3001202d8964b52a96b5337df3290d3"
format_stamp: "Formatted at 2026-08-08 00:41:58 on dist-test-slave-gjw7"
server_key: "a7be59076509d4db148ed15bc16be8ab"
server_key_iv: "ed63ed55c097092f7e398e9f8bda9497"
server_key_version: "encryptionkey@0"
I20260808 00:41:58.726864 8706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 00:41:58.732676 8693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } attempt: 2
I20260808 00:41:58.733055 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.732910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57948)
0808 00:41:58.732974 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.733017 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 00:41:58.733359 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.733273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57962)
0808 00:41:58.733320 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.733342 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 00:41:58.733440 8693 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.208.60:39041: Network error: Client connection negotiation failed: client connection to 127.1.208.60:39041: connect: Connection refused (error 111)
I20260808 00:41:58.741355 8706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 00:41:58.743454 8706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 00:41:58.743584 8706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 00:41:58.747519 8706 rpc_server.cc:307] RPC server started. Bound to: 127.1.208.60:39041
I20260808 00:41:58.747550 8829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.208.60:39041 every 8 connection(s)
I20260808 00:41:58.747892 8706 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/data/info.pb
I20260808 00:41:58.748850 8832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 00:41:58.750450 8832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.752539 1856 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasknSGkLY/build/release/bin/kudu as pid 8706
I20260808 00:41:58.752657 1856 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasknSGkLY/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786149440113891-1856-0/minicluster-data/master-2/wal/instance
I20260808 00:41:58.752853 1856 external_mini_cluster.cc:1428] Setting key 8d94732d4f23fef13ea4fb71eb41c281
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:58.760087 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.751135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60588 (local address 127.1.208.62:44143)
0808 00:41:58.751191 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.751193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.751384 (+ 191us) server_negotiation.cc:407] Connection header received
0808 00:41:58.751862 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.751863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.751875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.751924 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.752524 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.753107 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.754756 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.755070 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.755140 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.757558 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.757559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.757559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.757567 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.758823 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.759197 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.759199 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.759199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.759256 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.759477 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.759477 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.759478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.759590 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.759645 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.759904 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.759989 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 00:41:58.760457 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.751091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.62:44143 (local address 127.0.0.1:60588)
0808 00:41:58.751270 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.751306 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.751765 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.751993 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.751997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.752241 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.752437 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.752441 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.753458 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.753459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.754668 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.754671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.755179 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.755180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.755345 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.755831 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.755841 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.757480 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.758901 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.758908 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.758910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.759129 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.759311 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.759312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.759312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.759420 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.759645 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.759652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.759785 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.759874 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.760077 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":85,"threads_started":1}
I20260808 00:41:58.760941 8832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.761152 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.753268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57972 (local address 127.1.208.60:39041)
0808 00:41:58.753465 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.753469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.753486 (+ 17us) server_negotiation.cc:407] Connection header received
0808 00:41:58.753506 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.753508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.753522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.753583 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.754363 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.755231 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.756157 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.756449 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.756514 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.758688 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.758695 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.758696 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.758702 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.760091 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.760396 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.760399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.760399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.760451 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.760612 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.760613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.760613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.760771 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.760911 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.760982 (+ 71us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.761055 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":129,"threads_started":1}
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.62@KRBTEST.COM
I20260808 00:41:58.766692 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.761169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.61:34983 (local address 127.0.0.1:42932)
0808 00:41:58.761206 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.761214 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.761258 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.761410 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.761411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.761522 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.761549 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.761550 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.762206 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.762207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.762676 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.762678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.763008 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.763009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.763063 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.763390 (+ 327us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.763394 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.764394 (+ 1000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.765822 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.765823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.765824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.765976 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.766145 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.766145 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.766146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.766202 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.766415 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.766416 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.766456 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.766516 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.766602 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260808 00:41:58.766700 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.761226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.1.208.61:34983)
0808 00:41:58.761267 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.761269 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.761277 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:58.761341 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.761342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.761353 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.761401 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.761624 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.762139 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.762750 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.762951 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.763003 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.764466 (+ 1463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.764467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.764468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.764475 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.765752 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.766036 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.766037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.766038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.766093 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.766258 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.766259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.766259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.766362 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.766416 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.766545 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.766614 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260808 00:41:58.767020 8832 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } has no permanent_uuid. Determining permanent_uuid...
I20260808 00:41:58.768141 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.761939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.1.208.62:44143)
0808 00:41:58.761979 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.761981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.761988 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:58.762006 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.762007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.762015 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.762057 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.762453 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.762881 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.763483 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.763676 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.763726 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.765761 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.765762 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.765763 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.765769 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.767139 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.767485 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.767486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.767486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.767537 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.767697 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.767699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.767699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.767809 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.767957 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.767992 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.768056 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.60@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.208.61@KRBTEST.COM
I20260808 00:41:58.773310 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.767291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57978 (local address 127.1.208.60:39041)
0808 00:41:58.767353 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.767355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.767362 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:58.767381 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.767382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.767397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.767467 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.767794 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.768286 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.769095 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.769335 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.769393 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.770903 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.770904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.770905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.770911 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.772344 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.772646 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.772647 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.772647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.772693 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.772870 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.772871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.772871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.772955 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.773011 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.773136 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.773204 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 00:41:58.773315 8834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.767261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57978)
0808 00:41:58.767289 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.767297 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.767332 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.767594 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.767594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.767693 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.767728 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.767729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.768358 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.768359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.769017 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.769019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.769392 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.769393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.769431 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.769757 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.769761 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.770807 (+ 1046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.772421 (+ 1614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.772422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.772423 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.772574 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.772748 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.772749 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.772749 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.772811 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.773012 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.773013 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.773047 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.773096 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.773157 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13}
I20260808 00:41:58.774421 8571 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } attempt: 2
I20260808 00:41:58.775159 8832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: Bootstrap starting.
I20260808 00:41:58.775417 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.768709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.1.208.61:34983)
0808 00:41:58.768741 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.768743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.768751 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:58.768798 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.768799 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.768805 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.768852 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.769337 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.769963 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.770649 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.770843 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.770902 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.772989 (+ 2087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.772991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.772991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.772997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.774450 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.774732 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.774733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.774733 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.774784 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.774939 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.774940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.774940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.775057 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.775210 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.775264 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.775335 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260808 00:41:58.776177 8832 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:58.776451 8832 log.cc:824] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:58.776808 1856 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 00:41:58.777247 8832 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: No bootstrap required, opened a new log
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.62@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:58.778772 8832 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.778882 8832 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3001202d8964b52a96b5337df3290d3, State: Initialized, Role: FOLLOWER
I20260808 00:41:58.777942 8523 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "ffda26169fca4d479df6337f4ed25c8a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60596
I20260808 00:41:58.778932 8832 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.779187 8832 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:58.779187 8842 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.779325 8842 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:41:58.780917 8858 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:58.780987 8858 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:58.781790 8576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.774627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:57980)
0808 00:41:58.774718 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.774727 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.774762 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.775009 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.775011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.775153 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.775194 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.775196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.775982 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.775983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.776706 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.776710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.777100 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.777101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.777160 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.777539 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.777543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.778791 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.780541 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.780542 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.780542 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.780759 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.781076 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.781078 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.781078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.781152 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.781508 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.781509 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.781554 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.781611 (+ 57us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.781701 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 00:41:58.782228 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.774838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57980 (local address 127.1.208.60:39041)
0808 00:41:58.774872 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.774874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.774880 (+ 6us) server_negotiation.cc:407] Connection header received
0808 00:41:58.774897 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.774898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.774905 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.774980 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.775275 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.775898 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.776789 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.777028 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.777085 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.778881 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.778882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.778882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.778889 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.780419 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.780910 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.780911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.780911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.781008 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.781220 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.781221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.781221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.781412 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.781477 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.782087 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.782158 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260808 00:41:58.783599 8571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a: Bootstrap starting.
I20260808 00:41:58.781391 8706 master_runner.cc:186] Error getting master registration for 127.1.208.62:44143: OK, instance_id { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" instance_seqno: 1786149718433867 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:41:58.784082 8706 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:41:58.784303 8571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:58.784633 8571 log.cc:824] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:58.785470 8571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a: No bootstrap required, opened a new log
I20260808 00:41:58.786809 8571 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.786947 8571 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffda26169fca4d479df6337f4ed25c8a, State: Initialized, Role: FOLLOWER
I20260808 00:41:58.787030 8571 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.787302 8863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.787341 8571 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:58.787356 8523 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 00:41:58.787472 8523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:58.787432 8863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:58.788061 8523 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.788271 8523 leader_election.cc:288] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [CANDIDATE]: Term 1 election: Requested vote from peers 3bc4e16f761742f59b4300279bfcad69 (127.1.208.61:34983), d3001202d8964b52a96b5337df3290d3 (127.1.208.60:39041)
I20260808 00:41:58.788576 8788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffda26169fca4d479df6337f4ed25c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d3001202d8964b52a96b5337df3290d3"
I20260808 00:41:58.788676 8788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:58.790014 8788 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffda26169fca4d479df6337f4ed25c8a in term 1.
I20260808 00:41:58.790230 8485 leader_election.cc:302] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d3001202d8964b52a96b5337df3290d3, ffda26169fca4d479df6337f4ed25c8a; no voters:
I20260808 00:41:58.790710 8863 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 00:41:58.790892 8863 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [term 1 LEADER]: Becoming Leader. State: Replica: ffda26169fca4d479df6337f4ed25c8a, State: Running, Role: LEADER
I20260808 00:41:58.791042 8457 master_runner.cc:186] Error getting master registration for 127.1.208.61:34983: OK, instance_id { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" instance_seqno: 1786149718592576 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 00:41:58.793788 8457 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 00:41:58.793926 8863 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.794389 8863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffda26169fca4d479df6337f4ed25c8a. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.794485 8863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:58.795861 8896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 00:41:58.797310 8896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 00:41:58.800452 8698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.790694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42962 (local address 127.1.208.61:34983)
0808 00:41:58.790780 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.790781 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.791670 (+ 889us) server_negotiation.cc:407] Connection header received
0808 00:41:58.791791 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.791792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.791802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.791839 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.792910 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.793632 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.795156 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.795676 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.795737 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.797339 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.797341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.797341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.797348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.798913 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.799489 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.799491 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.799491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.799549 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.799767 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.799768 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.799768 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.799893 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.799984 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.800293 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.800364 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 00:41:58.802453 8578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.789787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60606 (local address 127.1.208.62:44143)
0808 00:41:58.789842 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.789844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.790540 (+ 696us) server_negotiation.cc:407] Connection header received
0808 00:41:58.790629 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.790630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.790640 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.790666 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.791549 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.792264 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.793261 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.793539 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.793580 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.795122 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.795123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.795124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.795130 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.796628 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.800607 (+ 3979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.800608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.800608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.800668 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.801864 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.801865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.801866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.801990 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.802070 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.802301 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.802374 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 00:41:58.803398 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.793861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57994 (local address 127.1.208.60:39041)
0808 00:41:58.793900 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.793901 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.795342 (+ 1441us) server_negotiation.cc:407] Connection header received
0808 00:41:58.795365 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.795365 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.795376 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.795422 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.797091 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.797782 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.798709 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.798998 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.799034 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.800344 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.800346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.800346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.800352 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.801759 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.802248 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.802261 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.802262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.802318 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.802808 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.802810 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.802810 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.802938 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.803005 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.803252 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.803318 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260808 00:41:58.812052 8788 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [term 1 FOLLOWER]: Refusing update from remote peer ffda26169fca4d479df6337f4ed25c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:58.812366 8863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:58.813862 8842 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffda26169fca4d479df6337f4ed25c8a. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.813930 8842 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:58.815022 8890 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.815088 8842 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.815126 8890 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: This master's current role is: LEADER
I20260808 00:41:58.815148 8842 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:58.815361 8900 mvcc.cc:205] Tried to move back new op lower bound from 7316069248207286272 to 7316069248180744192. Current Snapshot: MvccSnapshot[applied={T|T < 7316069248207286272}]
I20260808 00:41:58.821723 8904 mvcc.cc:205] Tried to move back new op lower bound from 7316069248207286272 to 7316069248180744192. Current Snapshot: MvccSnapshot[applied={T|T < 7316069248207286272}]
I20260808 00:41:58.822252 8896 catalog_manager.cc:1347] Generated new cluster ID: 76096a2820ed4ea781a3fa95d28ed073
I20260808 00:41:58.822304 8896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 00:41:58.824579 8693 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } attempt: 3
Aug 08 00:41:58 dist-test-slave-gjw7 krb5kdc[8435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786149718, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.208.61@KRBTEST.COM for kudu/127.1.208.60@KRBTEST.COM
I20260808 00:41:58.832158 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.824822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.208.60:39041 (local address 127.0.0.1:58004)
0808 00:41:58.824912 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0808 00:41:58.824923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 00:41:58.824961 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 00:41:58.825118 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 00:41:58.825119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 00:41:58.825338 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0808 00:41:58.825388 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.825390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.826269 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.826270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.827078 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 00:41:58.827081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.827561 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.827562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 00:41:58.827618 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.828023 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 00:41:58.828028 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 00:41:58.829521 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 00:41:58.831155 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.831156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.831156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.831367 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.831554 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 00:41:58.831555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 00:41:58.831555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0808 00:41:58.831609 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 00:41:58.831862 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 00:41:58.831863 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 00:41:58.831906 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 00:41:58.831962 (+ 56us) client_negotiation.cc:240] Negotiation successful
0808 00:41:58.832056 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 00:41:58.832499 8837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:58.824920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58004 (local address 127.1.208.60:39041)
0808 00:41:58.824955 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.824957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.824965 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:58.825031 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.825032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.825041 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.825081 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.825469 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.826183 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.827165 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:58.827450 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:58.827507 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:58.829609 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 00:41:58.829610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 00:41:58.829611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 00:41:58.829617 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0808 00:41:58.831067 (+ 1450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.831439 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.831440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.831441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.831495 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 00:41:58.831665 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 00:41:58.831665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 00:41:58.831666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 00:41:58.831783 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 00:41:58.831837 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:58.832338 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 00:41:58.832412 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260808 00:41:58.834050 8693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: Bootstrap starting.
I20260808 00:41:58.834762 8693 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: Neither blocks nor log segments found. Creating new log.
I20260808 00:41:58.835222 8693 log.cc:824] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: Log is configured to *not* fsync() on all Append() calls
I20260808 00:41:58.837057 8693 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: No bootstrap required, opened a new log
I20260808 00:41:58.838894 8693 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.838995 8693 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc4e16f761742f59b4300279bfcad69, State: Initialized, Role: FOLLOWER
I20260808 00:41:58.839044 8693 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } }
I20260808 00:41:58.839330 8693 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 00:41:58.839290 8920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.839385 8920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 00:41:58.840426 8619 tablet.cc:2369] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 00:41:58.840598 8655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffda26169fca4d479df6337f4ed25c8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3bc4e16f761742f59b4300279bfcad69"
I20260808 00:41:58.840665 8655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [term 0 FOLLOWER]: Advancing to term 1
I20260808 00:41:58.841259 8655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffda26169fca4d479df6337f4ed25c8a in term 1.
I20260808 00:41:58.841725 8654 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [term 1 FOLLOWER]: Refusing update from remote peer ffda26169fca4d479df6337f4ed25c8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 00:41:58.842417 8863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 00:41:58.842732 8896 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 00:41:58.843446 8896 catalog_manager.cc:1504] Loading token signing keys...
I20260808 00:41:58.844043 8920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffda26169fca4d479df6337f4ed25c8a. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.844113 8920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:58.844138 8920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.844168 8920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 00:41:58.847719 8896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a: Generated new TSK 0
I20260808 00:41:58.847939 8896 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 00:41:58.849874 8932 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 00:41:58.850937 8932 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 00:41:58.850406 8933 mvcc.cc:205] Tried to move back new op lower bound from 7316069248374382592 to 7316069248180744192. Current Snapshot: MvccSnapshot[applied={T|T < 7316069248207286272}]
I20260808 00:41:58.851125 8863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffda26169fca4d479df6337f4ed25c8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffda26169fca4d479df6337f4ed25c8a" member_type: VOTER last_known_addr { host: "127.1.208.62" port: 44143 } } peers { permanent_uuid: "3bc4e16f761742f59b4300279bfcad69" member_type: VOTER last_known_addr { host: "127.1.208.61" port: 34983 } } peers { permanent_uuid: "d3001202d8964b52a96b5337df3290d3" member_type: VOTER last_known_addr { host: "127.1.208.60" port: 39041 } } }
I20260808 00:41:58.851215 8863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffda26169fca4d479df6337f4ed25c8a [sys.catalog]: This master's current role is: LEADER
W20260808 00:41:58.878475 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.877056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.1.208.61:34983)
0808 00:41:58.877094 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.877095 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.877797 (+ 702us) server_negotiation.cc:407] Connection header received
0808 00:41:58.877902 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.877903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.877914 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.877943 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.878404 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42972: BlockingRecv error: recv got EOF from 127.0.0.1:42972 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260808 00:41:58.879540 8578 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.876838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60608 (local address 127.1.208.62:44143)
0808 00:41:58.876885 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.876887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.878881 (+ 1994us) server_negotiation.cc:407] Connection header received
0808 00:41:58.878981 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.878991 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.879016 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.879044 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.879473 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60608: BlockingRecv error: recv got EOF from 127.0.0.1:60608 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 00:41:58.880524 8837 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.877474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58020 (local address 127.1.208.60:39041)
0808 00:41:58.877523 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.877525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.879885 (+ 2360us) server_negotiation.cc:407] Connection header received
0808 00:41:58.879977 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.879978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.879990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.880018 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.880467 (+ 449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58020: BlockingRecv error: recv got EOF from 127.0.0.1:58020 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260808 00:41:58.882876 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.882120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.1.208.61:34983)
0808 00:41:58.882152 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.882154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.882375 (+ 221us) server_negotiation.cc:407] Connection header received
0808 00:41:58.882476 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.882478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.882496 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.882523 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.882829 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42988: BlockingRecv error: recv got EOF from 127.0.0.1:42988 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 00:41:58.885955 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.885191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.1.208.61:34983)
0808 00:41:58.885234 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.885236 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.885508 (+ 272us) server_negotiation.cc:407] Connection header received
0808 00:41:58.885596 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.885597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.885607 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.885633 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.885914 (+ 281us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42992: BlockingRecv error: recv got EOF from 127.0.0.1:42992 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260808 00:41:58.889942 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.889133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42994 (local address 127.1.208.61:34983)
0808 00:41:58.889175 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.889177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.889492 (+ 315us) server_negotiation.cc:407] Connection header received
0808 00:41:58.889580 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.889581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.889590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.889617 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.889899 (+ 282us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42994: BlockingRecv error: recv got EOF from 127.0.0.1:42994 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 00:41:58.895835 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.895091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43010 (local address 127.1.208.61:34983)
0808 00:41:58.895124 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.895126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.895357 (+ 231us) server_negotiation.cc:407] Connection header received
0808 00:41:58.895446 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.895447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.895457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.895484 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.895786 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43010: BlockingRecv error: recv got EOF from 127.0.0.1:43010 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260808 00:41:58.905915 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.905125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.1.208.61:34983)
0808 00:41:58.905160 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.905162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.905425 (+ 263us) server_negotiation.cc:407] Connection header received
0808 00:41:58.905515 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.905517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.905527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.905554 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.905866 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43018: BlockingRecv error: recv got EOF from 127.0.0.1:43018 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260808 00:41:58.924412 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.923450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43026 (local address 127.1.208.61:34983)
0808 00:41:58.923494 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.923497 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.923800 (+ 303us) server_negotiation.cc:407] Connection header received
0808 00:41:58.923908 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.923909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.923922 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.923951 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.924354 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43026: BlockingRecv error: recv got EOF from 127.0.0.1:43026 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260808 00:41:58.959143 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:58.958058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.1.208.61:34983)
0808 00:41:58.958108 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:58.958110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:58.958452 (+ 342us) server_negotiation.cc:407] Connection header received
0808 00:41:58.958571 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:58.958572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:58.958586 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:58.958629 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:58.959065 (+ 436us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43034: BlockingRecv error: recv got EOF from 127.0.0.1:43034 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260808 00:41:59.025425 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:59.024676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43038 (local address 127.1.208.61:34983)
0808 00:41:59.024728 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.024731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.024841 (+ 110us) server_negotiation.cc:407] Connection header received
0808 00:41:59.024911 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.024912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.024925 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.024974 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:59.025359 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43038: BlockingRecv error: recv got EOF from 127.0.0.1:43038 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260808 00:41:59.155267 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:59.154567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43040 (local address 127.1.208.61:34983)
0808 00:41:59.154646 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.154648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.154727 (+ 79us) server_negotiation.cc:407] Connection header received
0808 00:41:59.154807 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.154808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.154821 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.154871 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:59.155216 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43040: BlockingRecv error: recv got EOF from 127.0.0.1:43040 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260808 00:41:59.413142 8698 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 00:41:59.412511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.1.208.61:34983)
0808 00:41:59.412585 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.412587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.412647 (+ 60us) server_negotiation.cc:407] Connection header received
0808 00:41:59.412700 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.412701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.412711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.412771 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 00:41:59.413087 (+ 316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43048: BlockingRecv error: recv got EOF from 127.0.0.1:43048 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 00:41:59.781443 8858 catalog_manager.cc:1259] Loaded cluster ID: 76096a2820ed4ea781a3fa95d28ed073
I20260808 00:41:59.781517 8858 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: loading cluster ID for follower catalog manager: success
I20260808 00:41:59.782289 8858 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: acquiring CA information for follower catalog manager: success
I20260808 00:41:59.782544 8858 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3001202d8964b52a96b5337df3290d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:41:59.851487 8932 catalog_manager.cc:1259] Loaded cluster ID: 76096a2820ed4ea781a3fa95d28ed073
I20260808 00:41:59.851567 8932 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: loading cluster ID for follower catalog manager: success
I20260808 00:41:59.852339 8932 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: acquiring CA information for follower catalog manager: success
I20260808 00:41:59.852624 8932 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3bc4e16f761742f59b4300279bfcad69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 00:41:59.928835 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:59.926253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33764 (local address 127.1.208.61:34983)
0808 00:41:59.926392 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.926394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.926463 (+ 69us) server_negotiation.cc:407] Connection header received
0808 00:41:59.926540 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.926541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.926601 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.926676 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:59.926822 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.927327 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.928067 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.928266 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.928322 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:59.928484 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:59.928596 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:59.928662 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:59.928710 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 00:41:59.928731 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
I20260808 00:41:59.932773 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:59.930518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39228 (local address 127.1.208.60:39041)
0808 00:41:59.930646 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.930648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.930656 (+ 8us) server_negotiation.cc:407] Connection header received
0808 00:41:59.930699 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.930700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.930743 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.930805 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:59.930905 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.931384 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.932076 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.932284 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.932342 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:59.932472 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:59.932574 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:59.932634 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:59.932673 (+ 39us) server_negotiation.cc:299] Negotiation successful
0808 00:41:59.932693 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":53,"threads_started":1}
I20260808 00:41:59.936759 9064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:59.934332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.1.208.61:34983)
0808 00:41:59.934458 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.934460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.934469 (+ 9us) server_negotiation.cc:407] Connection header received
0808 00:41:59.934497 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.934498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.934507 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.934565 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:59.934688 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.935270 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936071 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.936364 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936428 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:59.936546 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:59.936576 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:59.936618 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:59.936667 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 00:41:59.936687 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260808 00:41:59.937371 9070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:59.934537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.1.208.60:39041)
0808 00:41:59.934572 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.934573 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.934726 (+ 153us) server_negotiation.cc:407] Connection header received
0808 00:41:59.934748 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.934749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.934757 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.934807 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:59.935040 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.935724 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936679 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.936932 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936984 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:59.937143 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:59.937173 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:59.937213 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:59.937264 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 00:41:59.937283 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260808 00:41:59.937458 9083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 00:41:59.934357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46870 (local address 127.1.208.62:44143)
0808 00:41:59.934634 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 00:41:59.934636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0808 00:41:59.934643 (+ 7us) server_negotiation.cc:407] Connection header received
0808 00:41:59.934687 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 00:41:59.934688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 00:41:59.934740 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 00:41:59.934812 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 00:41:59.934927 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.935676 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936505 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 00:41:59.936826 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 00:41:59.936884 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 00:41:59.937094 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 00:41:59.937196 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 00:41:59.937259 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 00:41:59.937345 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 00:41:59.937374 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":58,"threads_started":1}
I20260808 00:41:59.941856 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8457
I20260808 00:41:59.959764 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8584
I20260808 00:41:59.968719 1856 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasknSGkLY/build/release/bin/kudu with pid 8706
2026-08-08T00:41:59Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2956 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5901 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (279866 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST