U5Mqy4WY

1============================= test session starts ==============================
2platform darwin -- Python 3.7.1, pytest-3.6.4, py-1.8.0, pluggy-0.7.1
3rootdir: /Users/asp/development/cassandra-dtest, inifile: pytest.ini
4plugins: timeout-1.3.3, flaky-3.6.1
5timeout: 900.0s
6timeout method: signal
7timeout func_only: False
8collected 5529 items / 4485 deselected
9
10auditlog_test.py sssss                                                   [  0%]
11auth_join_ring_false_test.py ....                                        [  0%]
12auth_test.py ..ss....s.................................................. [  6%]
13......................ss....ssssss                                       [  9%]
14batch_test.py .........................                                  [ 12%]
15bootstrap_test.py .....................F                                 [ 14%]
16cdc_test.py .....                                                        [ 14%]
17cfid_test.py .                                                           [ 14%]
18commitlog_test.py ............                                           [ 15%]
19compaction_test.py sss..........................s                        [ 18%]
20compression_test.py ....                                                 [ 19%]
21concurrent_schema_changes_test.py sssssssssssss                          [ 20%]
22configuration_test.py ..                                                 [ 20%]
23consistency_test.py ..............F....                                  [ 22%]
24consistent_bootstrap_test.py ..                                          [ 22%]
25counter_test.py ........                                                 [ 23%]
26cql_prepared_test.py .                                                   [ 23%]
27cql_test.py .........sF....s..........                                   [ 26%]
28cql_tracing_test.py ....                                                 [ 26%]
29delete_insert_test.py .                                                  [ 26%]
30deletion_test.py ..                                                      [ 26%]
31disk_balance_test.py ..........                                          [ 27%]
32fqltool_test.py sss                                                      [ 27%]
33global_row_key_cache_test.py .                                           [ 28%]
34hintedhandoff_test.py ...s...                                            [ 28%]
35internode_ssl_test.py ..                                                 [ 28%]
36jmx_auth_test.py .                                                       [ 29%]
37jmx_test.py .....s..                                                     [ 29%]
38json_test.py ...............                                             [ 31%]
39json_tools_test.py s                                                     [ 31%]
40largecolumn_test.py F                                                    [ 31%]
41legacy_sstables_test.py .                                                [ 31%]
42materialized_views_test.py .........................sss...s............. [ 35%]
43..s..                                                                    [ 36%]
44metadata_test.py s                                                       [ 36%]
45mixed_version_test.py s                                                  [ 36%]
46multidc_putget_test.py ..                                                [ 36%]
47native_transport_ssl_test.py ....                                        [ 37%]
48nodetool_test.py ......s..ss                                             [ 38%]
49offline_tools_test.py ..F...                                             [ 38%]
50paging_test.py ...................................................s..... [ 44%]
51                                                                         [ 44%]
52paxos_test.py ..s.                                                       [ 44%]
53prepared_statements_test.py .                                            [ 44%]
54pushed_notifications_test.py .......                                     [ 45%]
55putget_test.py .......                                                   [ 45%]
56range_ghost_test.py .                                                    [ 46%]
57read_failures_test.py ...                                                [ 46%]
58read_repair_test.py ....ssssssssss                                       [ 47%]
59rebuild_test.py ......                                                   [ 48%]
60refresh_test.py .                                                        [ 48%]
61replace_address_test.py .....                                            [ 48%]
62replication_test.py .....ss...s.                                         [ 50%]
63schema_metadata_test.py ..................                               [ 51%]
64schema_test.py ......                                                    [ 52%]
65scrub_test.py .......                                                    [ 52%]
66secondary_indexes_test.py .......sss.......s....                         [ 55%]
67seed_test.py ...                                                         [ 55%]
68snapshot_test.py ..........                                              [ 56%]
69snitch_test.py .                                                         [ 56%]
70sslnodetonode_test.py .......s                                           [ 57%]
71sstable_generation_loading_test.py sssssssssssssssssss....s              [ 59%]
72sstablesplit_test.py ..                                                  [ 59%]
73sstableutil_test.py ..                                                   [ 59%]
74streaming_test.py ssss                                                   [ 60%]
75stress_tool_test.py ...                                                  [ 60%]
76super_column_cache_test.py .                                             [ 60%]
77super_counter_test.py .                                                  [ 60%]
78system_keyspaces_test.py ..                                              [ 60%]
79thrift_hsha_test.py .s                                                   [ 61%]
80thrift_test.py ........................s................................ [ 66%]
81...................................................                      [ 71%]
82token_generator_test.py sss                                              [ 71%]
83topology_test.py ...s......                                              [ 72%]
84transient_replication_ring_test.py ssssssss                              [ 73%]
85transient_replication_test.py ssssssssssssssssss                         [ 75%]
86ttl_test.py ..........s..............                                    [ 77%]
87udtencoding_test.py .                                                    [ 77%]
88user_functions_test.py ........                                          [ 78%]
89user_types_test.py ............                                          [ 79%]
90wide_rows_test.py ..                                                     [ 79%]
91write_failures_test.py s..s........                                      [ 80%]
92cqlsh_tests/test_cqlsh.py ............................................s. [ 85%]
93.......                                                                  [ 86%]
94cqlsh_tests/test_cqlsh_copy.py ......................................... [ 89%]
95.......                                                                  [ 90%]
96meta_tests/assertion_test.py ...........                                 [ 91%]
97meta_tests/utils_test/funcutils_test.py ...........                      [ 92%]
98meta_tests/utils_test/metadata_wrapper_test.py ...............           [ 94%]
99repair_tests/deprecated_repair_test.py .......                           [ 94%]
100repair_tests/incremental_repair_test.py ssss.....sssssssssss             [ 96%]
101repair_tests/preview_repair_test.py s                                    [ 96%]
102repair_tests/repair_test.py ...........................s.....            [100%]
103
104=================================== FAILURES ===================================
105_________________ TestBootstrap.test_bootstrap_binary_disabled _________________
106
107self = <bootstrap_test.TestBootstrap object at 0x111ce1080>
108node = <ccmlib.node.Node object at 0x111e79ac8>
109msg = 'Not starting client transports as bootstrap has not completed'
110timeout = 30, kwargs = {}
111
112    def assert_log_had_msg(self, node, msg, timeout=600, **kwargs):
113        """
114            Wrapper for ccmlib.node.Node#watch_log_for to cause an assertion failure when a log message isn't found
115            within the timeout.
116            :param node: Node which logs we should watch
117            :param msg: String message we expect to see in the logs.
118            :param timeout: Seconds to wait for msg to appear
119            """
120        try:
121>           node.watch_log_for(msg, timeout=timeout, **kwargs)
122
123dtest.py:269: 
124_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
125
126self = <ccmlib.node.Node object at 0x111e79ac8>
127exprs = 'Not starting client transports as bootstrap has not completed'
128from_mark = None, timeout = 30, process = None, verbose = False
129filename = 'system.log'
130
131    def watch_log_for(self, exprs, from_mark=None, timeout=600, process=None, verbose=False, filename='system.log'):
132        """
133            Watch the log until one or more (regular) expression are found.
134            This methods when all the expressions have been found or the method
135            timeouts (a TimeoutError is then raised). On successful completion,
136            a list of pair (line matched, match object) is returned.
137            """
138        start = time.time()
139        tofind = [exprs] if isinstance(exprs, string_types) else exprs
140        tofind = [re.compile(e) for e in tofind]
141        matchings = []
142        reads = ""
143        if len(tofind) == 0:
144            return None
145    
146        log_file = os.path.join(self.get_path(), 'logs', filename)
147        output_read = False
148        while not os.path.exists(log_file):
149            time.sleep(.5)
150            if start + timeout < time.time():
151                raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", time.gmtime()) + " [" + self.name + "] Timed out waiting for {} to be created.".format(log_file))
152            if process and not output_read:
153                process.poll()
154                if process.returncode is not None:
155                    self.print_process_output(self.name, process, verbose)
156                    output_read = True
157                    if process.returncode != 0:
158                        raise RuntimeError()  # Shouldn't reuse RuntimeError but I'm lazy
159    
160        with open(log_file) as f:
161            if from_mark:
162                f.seek(from_mark)
163    
164            while True:
165                # First, if we have a process to check, then check it.
166                # Skip on Windows - stdout/stderr is cassandra.bat
167                if not common.is_win() and not output_read:
168                    if process:
169                        process.poll()
170                        if process.returncode is not None:
171                            self.print_process_output(self.name, process, verbose)
172                            output_read = True
173                            if process.returncode != 0:
174                                raise RuntimeError()  # Shouldn't reuse RuntimeError but I'm lazy
175    
176                line = f.readline()
177                if line:
178                    reads = reads + line
179                    for e in tofind:
180                        m = e.search(line)
181                        if m:
182                            matchings.append((line, m))
183                            tofind.remove(e)
184                            if len(tofind) == 0:
185                                return matchings[0] if isinstance(exprs, string_types) else matchings
186                else:
187                    # yep, it's ugly
188                    time.sleep(1)
189                    if start + timeout < time.time():
190>                       raise TimeoutError(time.strftime("%d %b %Y %H:%M:%S", time.gmtime()) + " [" + self.name + "] Missing: " + str([e.pattern for e in tofind]) + ":\n" + reads[:50] + ".....\nSee {} for remainder".format(filename))
191E                       ccmlib.node.TimeoutError: 26 Dec 2019 22:42:43 [node2] Missing: ['Not starting client transports as bootstrap has not completed']:
192E                       INFO  [main] 2019-12-26 14:42:00,604 YamlConfigura.....
193E                       See system.log for remainder
194
195../../dtest/src/ccm/ccmlib/node.py:536: TimeoutError
196
197During handling of the above exception, another exception occurred:
198
199self = <bootstrap_test.TestBootstrap object at 0x111ce1080>
200
201    @since('2.2')
202    def test_bootstrap_binary_disabled(self):
203        """
204            Test binary while bootstrapping and streaming fails
205            @jira_ticket CASSANDRA-14526, CASSANDRA-14525
206            """
207        config = {'authenticator': 'org.apache.cassandra.auth.PasswordAuthenticator',
208                  'authorizer': 'org.apache.cassandra.auth.CassandraAuthorizer',
209                  'role_manager': 'org.apache.cassandra.auth.CassandraRoleManager',
210                  'permissions_validity_in_ms': 0,
211                  'roles_validity_in_ms': 0}
212    
213        cluster = self.cluster
214        cluster.populate(1)
215    
216        node1 = cluster.nodes['node1']
217        # set up byteman
218        node1.byteman_port = '8100'
219        node1.import_config_files()
220    
221        cluster.start(wait_other_notice=True)
222        # kill stream to node2 in the middle of streaming to let it fail
223        if cluster.version() < '4.0':
224            node1.byteman_submit([self.byteman_submit_path_pre_4_0])
225        else:
226            node1.byteman_submit([self.byteman_submit_path_4_0])
227        node1.stress(['write', 'n=1K', 'no-warmup', 'cl=ONE', '-schema', 'replication(factor=3)', '-rate', 'threads=50', '-mode', 'native', 'cql3', 'user=cassandra', 'password=cassandra'])
228        cluster.flush()
229    
230        # start bootstrapping node2 and wait for streaming
231        node2 = new_node(cluster)
232        node2.set_configuration_options(values=config)
233        node2.byteman_port = '8101' # set for when we add node3
234        node2.import_config_files()
235        node2.start(jvm_args=["-Dcassandra.ring_delay_ms=5000"], wait_other_notice=True)
236        self.assert_log_had_msg(node2, 'Some data streaming failed', timeout=30)
237>       self.assert_log_had_msg(node2, 'Not starting client transports as bootstrap has not completed', timeout=30)
238
239bootstrap_test.py:763: 
240_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
241
242self = <bootstrap_test.TestBootstrap object at 0x111ce1080>
243node = <ccmlib.node.Node object at 0x111e79ac8>
244msg = 'Not starting client transports as bootstrap has not completed'
245timeout = 30, kwargs = {}
246
247    def assert_log_had_msg(self, node, msg, timeout=600, **kwargs):
248        """
249            Wrapper for ccmlib.node.Node#watch_log_for to cause an assertion failure when a log message isn't found
250            within the timeout.
251            :param node: Node which logs we should watch
252            :param msg: String message we expect to see in the logs.
253            :param timeout: Seconds to wait for msg to appear
254            """
255        try:
256            node.watch_log_for(msg, timeout=timeout, **kwargs)
257        except TimeoutError:
258>           pytest.fail("Log message was not seen within timeout:\n{0}".format(msg))
259E           Failed: Log message was not seen within timeout:
260E           Not starting client transports as bootstrap has not completed
261
262dtest.py:271: Failed
263---------------------------- Captured stdout setup -----------------------------
26414:41:40,379 ccm DEBUG Log-watching thread starting.
265------------------------------ Captured log setup ------------------------------
26614:41:40,299 conftest INFO Starting execution of test_bootstrap_binary_disabled at 2019-12-26 14:41:40.299365
26714:41:40,300 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-cofzrvve
268----------------------------- Captured stdout call -----------------------------
269install rule inject stream failure
270
271--------------------------- Captured stdout teardown ---------------------------
27214:42:43,653 ccm DEBUG Log-watching thread exiting.
273_______________________ TestConsistency.test_short_read ________________________
274
275self = <consistency_test.TestConsistency object at 0x11a40fb70>
276
277    def test_short_read(self):
278        """
279            @jira_ticket CASSANDRA-9460
280            """
281        cluster = self.cluster
282    
283        # this test causes the python driver to be extremely noisy due to
284        # frequent starting and stopping of nodes. let's move the log level
285        # of the driver to ERROR for this test only
286        logging.getLogger("cassandra").setLevel('ERROR')
287    
288        # Disable hinted handoff and set batch commit log so this doesn't
289        # interfer with the test
290        cluster.set_configuration_options(values={'hinted_handoff_enabled': False})
291        cluster.set_batch_commitlog(enabled=True)
292    
293        cluster.populate(3).start(wait_other_notice=True)
294        node1, node2, node3 = cluster.nodelist()
295    
296        session = self.patient_cql_connection(node1)
297        create_ks(session, 'ks', 3)
298    
299        if cluster.version() < '4.0':
300            create_cf(session, 'cf', read_repair=0.0)
301        else:
302            create_cf(session, 'cf')
303    
304        normal_key = 'normal'
305        reversed_key = 'reversed'
306    
307        # Repeat this test 10 times to make it more easy to spot a null pointer exception caused by a race, see CASSANDRA-9460
308        for k in range(10):
309            # insert 9 columns in two rows
310            insert_columns(self, session, normal_key, 9)
311            insert_columns(self, session, reversed_key, 9)
312    
313            # Delete 3 first columns (and 3 last columns, for the reversed version) with a different node dead each time
314            for node, column_number_to_delete in zip(list(range(1, 4)), list(range(3))):
315>               self.stop_node(node)
316
317/Users/asp/development/cassandra-dtest/consistency_test.py:1395: 
318_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
319/Users/asp/development/cassandra-dtest/consistency_test.py:1576: in stop_node
320    to_stop.flush()
321/Users/asp/dtest/src/ccm/ccmlib/node.py:1375: in flush
322    self.nodetool(cmd)
323/Users/asp/dtest/src/ccm/ccmlib/node.py:853: in nodetool
324    p = self.nodetool_process(cmd)
325/Users/asp/dtest/src/ccm/ccmlib/node.py:845: in nodetool_process
326    env = self.get_env()
327/Users/asp/dtest/src/ccm/ccmlib/node.py:205: in get_env
328    env = common.make_cassandra_env(self.get_install_dir(), self.get_path(), update_conf)
329/Users/asp/dtest/src/ccm/ccmlib/common.py:280: in make_cassandra_env
330    ???
331/Users/asp/dtest/lib/python3.7/shutil.py:241: in copy
332    ???
333_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
334
335src = '/Users/asp/development/cassandra/bin/cassandra.in.sh'
336dst = '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-4mfru3_j/test/node1/bin/cassandra.in.sh'
337
338>   ???
339E   OSError: [Errno 24] Too many open files: '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-4mfru3_j/test/node1/bin/cassandra.in.sh'
340
341/Users/asp/dtest/lib/python3.7/shutil.py:121: OSError
342---------------------------- Captured stdout setup -----------------------------
34315:23:36,796 ccm DEBUG Log-watching thread starting.
344------------------------------ Captured log setup ------------------------------
34515:23:36,719 conftest INFO Starting execution of test_short_read at 2019-12-26 15:23:36.719705
34615:23:36,720 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-4mfru3_j
347----------------------------- Captured stdout call -----------------------------
348
349----------------------------- Captured stderr call -----------------------------
350Exception in thread Thread-237:
351Traceback (most recent call last):
352  File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/threading.py", line 917, in _bootstrap_inner
353  File "/Users/asp/dtest/src/ccm/ccmlib/cluster.py", line 189, in run
354  File "/Users/asp/dtest/src/ccm/ccmlib/cluster.py", line 182, in scan_and_report
355  File "/Users/asp/development/cassandra-dtest/dtest_setup.py", line 137, in _log_error_handler
356  File "/Users/asp/dtest/lib/python3.7/site-packages/_pytest/outcomes.py", line 97, in fail
357Failed: Error details: 
358Errors seen in logs for: log_scanner
359log_scanner: [Errno 24] Too many open files: '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-4mfru3_j/test/node1/logs/system.log'
360
361_________________ TestMiscellaneousCQL.test_cql3_insert_thrift _________________
362
363self = <cql_test.TestMiscellaneousCQL object at 0x113928048>
364
365    @since('2.0', max_version='4')
366    def test_cql3_insert_thrift(self):
367        """
368            Check that we can insert from thrift into a CQL3 table:
369    
370            - CREATE a table via CQL
371            - insert values via thrift
372            - SELECT the inserted values and assert they are there as expected
373    
374            @jira_ticket CASSANDRA-4377
375            """
376        session = self.prepare(start_rpc=True)
377    
378        session.execute("""
379                CREATE TABLE test (
380                    k int,
381                    c int,
382                    v int,
383                    PRIMARY KEY (k, c)
384                )
385            """)
386    
387        node = self.cluster.nodelist()[0]
388        host, port = node.network_interfaces['thrift']
389        client = get_thrift_client(host, port)
390>       client.transport.open()
391
392cql_test.py:547: 
393_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
394../../dtest/lib/python3.7/site-packages/thrift/transport/TTransport.py:269: in open
395    return self.__trans.open()
396_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
397
398self = <thrift.transport.TSocket.TSocket object at 0x112cfba20>
399
400    def open(self):
401        if self.handle:
402            raise TTransportException(TTransportException.ALREADY_OPEN)
403        try:
404            addrs = self._resolveAddr()
405        except socket.gaierror:
406            msg = 'failed to resolve sockaddr for ' + str(self._address)
407            logger.exception(msg)
408            raise TTransportException(TTransportException.NOT_OPEN, msg)
409        for family, socktype, _, _, sockaddr in addrs:
410            handle = self._do_open(family, socktype)
411            handle.settimeout(self._timeout)
412            try:
413                handle.connect(sockaddr)
414                self.handle = handle
415                return
416            except socket.error:
417                handle.close()
418                logger.info('Could not connect to %s', sockaddr, exc_info=True)
419        msg = 'Could not connect to any of %s' % list(map(lambda a: a[4],
420                                                          addrs))
421        logger.error(msg)
422>       raise TTransportException(TTransportException.NOT_OPEN, msg)
423E       thrift.transport.TTransport.TTransportException: Could not connect to any of [('127.0.0.1', 9160)]
424
425../../dtest/lib/python3.7/site-packages/thrift/transport/TSocket.py:113: TTransportException
426---------------------------- Captured stdout setup -----------------------------
42715:46:35,568 ccm DEBUG Log-watching thread starting.
428------------------------------ Captured log setup ------------------------------
42915:46:35,497 conftest INFO Starting execution of test_cql3_insert_thrift at 2019-12-26 15:46:35.497315
43015:46:35,497 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-z9z5fb7l
431------------------------------ Captured log call -------------------------------
43215:46:41,282 thrift.transport.TSocket INFO Could not connect to ('127.0.0.1', 9160)
433Traceback (most recent call last):
434  File "/Users/asp/dtest/lib/python3.7/site-packages/thrift/transport/TSocket.py", line 104, in open
435    handle.connect(sockaddr)
436ConnectionRefusedError: [Errno 61] Connection refused
43715:46:41,285 thrift.transport.TSocket ERROR Could not connect to any of [('127.0.0.1', 9160)]
438--------------------------- Captured stdout teardown ---------------------------
43915:46:41,644 ccm DEBUG Log-watching thread exiting.
440_________________________ TestLargeColumn.test_cleanup _________________________
441
442self = <largecolumn_test.TestLargeColumn object at 0x112c57fd0>
443
444    def test_cleanup(self):
445        """
446            @jira_ticket CASSANDRA-8670
447            """
448        cluster = self.cluster
449        # Commit log segment size needs to increase for the database to be willing to accept columns that large
450        # internode compression is disabled because the regression being tested occurs in NIO buffer pooling without compression
451        configuration = {'commitlog_segment_size_in_mb': 128, 'internode_compression': 'none'}
452        if cluster.version() >= '4.0':
453            configuration['internode_max_message_size_in_bytes'] = 128 * 1024 * 1024
454        cluster.set_configuration_options(configuration)
455    
456        # Have Netty allocate memory on heap so it is clear if memory used for large columns is related to intracluster messaging
457        cluster.populate(2).start(jvm_args=[" -Dcassandra.netty_use_heap_allocator=true "])
458        node1, node2 = cluster.nodelist()
459    
460        session = self.patient_cql_connection(node1)
461        logger.debug("Before stress {0}".format(self.directbytes(node1)))
462        logger.debug("Running stress")
463        # Run the full stack to see how much memory is utilized for "small" columns
464        self.stress_with_col_size(cluster, node1, 1)
465        beforeStress = self.directbytes(node1)
466        logger.debug("Ran stress once {0}".format(beforeStress))
467    
468        # Now run the full stack to see how much memory is utilized for "large" columns
469        LARGE_COLUMN_SIZE = 1024 * 1024 * 63
470>       self.stress_with_col_size(cluster, node1, LARGE_COLUMN_SIZE)
471
472largecolumn_test.py:67: 
473_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
474largecolumn_test.py:20: in stress_with_col_size
475    node.stress(['write', 'n=5', "no-warmup", "cl=ALL", "-pop", "seq=1...5", "-schema", "replication(factor=2)", "-col", "n=fixed(1)", "size=fixed(" + size + ")", "-rate", "threads=1"])
476../../dtest/src/ccm/ccmlib/node.py:1348: in stress
477    return handle_external_tool_process(p, ['stress'] + stress_options)
478_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
479
480process = <subprocess.Popen object at 0x112c04978>
481cmd_args = ['stress', 'write', 'n=5', 'no-warmup', 'cl=ALL', '-pop', ...]
482
483    def handle_external_tool_process(process, cmd_args):
484        out, err = process.communicate()
485        if (out is not None) and isinstance(out, bytes):
486            out = out.decode()
487        if (err is not None) and isinstance(err, bytes):
488            err = err.decode()
489        rc = process.returncode
490    
491        if rc != 0:
492>           raise ToolError(cmd_args, rc, out, err)
493E           ccmlib.node.ToolError: Subprocess ['stress', 'write', 'n=5', 'no-warmup', 'cl=ALL', '-pop', 'seq=1...5', '-schema', 'replication(factor=2)', '-col', 'n=fixed(1)', 'size=fixed(66060288)', '-rate', 'threads=1'] exited with non-zero status; exit status: 1; 
494E           stdout: ******************** Stress Settings ********************
495E           Command:
496E             Type: write
497E             Count: 5
498E             No Warmup: true
499E             Consistency Level: ALL
500E             Target Uncertainty: not applicable
501E             Key Size (bytes): 10
502E             Counter Increment Distibution: add=fixed(1)
503E           Rate:
504E             Auto: false
505E             Thread Count: 1
506E             OpsPer Sec: 0
507E           Population:
508E             Sequence: 1..5
509E             Order: ARBITRARY
510E             Wrap: true
511E           Insert:
512E             Revisits: Uniform:  min=1,max=1000000
513E             Visits: Fixed:  key=1
514E             Row Population Ratio: Ratio: divisor=1.000000;delegate=Fixed:  key=1
515E             Batch Type: not batching
516E           Columns:
517E             Max Columns Per Key: 1
518E             Column Names: [C0]
519E             Comparator: AsciiType
520E             Timestamp: null
521E             Variable Column Count: false
522E             Slice: false
523E             Size Distribution: Fixed:  key=66060288
524E             Count Distribution: Fixed:  key=1
525E           Errors:
526E             Ignore: false
527E             Tries: 10
528E           Log:
529E             No Summary: false
530E             No Settings: false
531E             File: null
532E             Interval Millis: 1000
533E             Level: NORMAL
534E           Mode:
535E             API: JAVA_DRIVER_NATIVE
536E             Connection Style: CQL_PREPARED
537E             CQL Version: CQL3
538E             Protocol Version: V4
539E             Username: null
540E             Password: null
541E             Auth Provide Class: null
542E             Max Pending Per Connection: 128
543E             Connections Per Host: 8
544E             Compression: NONE
545E           Node:
546E             Nodes: [127.0.0.1]
547E             Is White List: false
548E             Datacenter: null
549E           Schema:
550E             Keyspace: keyspace1
551E             Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
552E             Replication Strategy Pptions: {replication_factor=2}
553E             Table Compression: null
554E             Table Compaction Strategy: null
555E             Table Compaction Strategy Options: {}
556E           Transport:
557E             factory=org.apache.cassandra.thrift.TFramedTransportFactory; truststore=null; truststore-password=null; keystore=null; keystore-password=null; ssl-protocol=TLS; ssl-alg=SunX509; store-type=JKS; ssl-ciphers=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA; 
558E           Port:
559E             Native Port: 9042
560E             Thrift Port: 9160
561E             JMX Port: 7100
562E           Send To Daemon:
563E             *not set*
564E           Graph:
565E             File: null
566E             Revision: unknown
567E             Title: null
568E             Operation: WRITE
569E           TokenRange:
570E             Wrap: false
571E             Split Factor: 1
572E           
573E           Connected to cluster: test, max pending requests per connection 128, max connections per host 8
574E           Datatacenter: datacenter1; Host: /127.0.0.1; Rack: rack1
575E           Datatacenter: datacenter1; Host: /127.0.0.2; Rack: rack1
576E           Created keyspaces. Sleeping 1s for propagation.
577E           Sleeping 2s...
578E           Running WRITE with 1 threads for 5 iteration
579E           type       total ops,    op/s,    pk/s,   row/s,    mean,     med,     .95,     .99,    .999,     max,   time,   stderr, errors,  gc: #,  max ms,  sum ms,  sdv ms,      mb
580E           java.io.IOException: Operation x10 on key(s) [30503337373039503231]: Error executing: (NoHostAvailableException): All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
581E           
582E           	at org.apache.cassandra.stress.Operation.error(Operation.java:136)
583E           	at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:114)
584E           	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
585E           	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
586E           	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:264)
587E           	at org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:472)
588E           total,             1,       1,       1,       1, 17758.7, 17767.1, 17767.1, 17767.1, 17767.1, 17767.1,   18.8, -0.97333,      1,      0,       0,       0,       0,       0
589E           FAILURE
590E           ; 
591E           stderr: Failed to connect over JMX; not collecting these stats
592E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.OperationTimedOutException: [localhost/127.0.0.1] Timed out waiting for server response), 127.0.0.2/127.0.0.2:9042 (com.datastax.driver.core.exceptions.TransportException: [127.0.0.2/127.0.0.2] Connection has been closed))
593E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
594E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
595E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
596E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
597E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
598E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
599E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
600E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
601E           com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
602E           java.lang.RuntimeException: Failed to execute stress action
603E           	at org.apache.cassandra.stress.StressAction.run(StressAction.java:100)
604E           	at org.apache.cassandra.stress.Stress.run(Stress.java:143)
605E           	at org.apache.cassandra.stress.Stress.main(Stress.java:62)
606
607../../dtest/src/ccm/ccmlib/node.py:2127: ToolError
608---------------------------- Captured stdout setup -----------------------------
60916:45:12,41 ccm DEBUG Log-watching thread starting.
610------------------------------ Captured log setup ------------------------------
61116:45:11,964 conftest INFO Starting execution of test_cleanup at 2019-12-26 16:45:11.964893
61216:45:11,966 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-c2pqwr17
613------------------------------ Captured log call -------------------------------
61416:45:29,256 cassandra.cluster INFO New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
61516:45:59,290 cassandra.cluster WARNING Host 127.0.0.2 has been marked down
61616:46:00,326 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 2.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
61716:46:02,381 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 4.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
61816:46:06,485 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 8.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
619--------------------------- Captured stdout teardown ---------------------------
62016:46:14,106 ccm DEBUG Log-watching thread exiting.
621---------------------------- Captured stdout setup -----------------------------
62216:46:15,465 ccm DEBUG Log-watching thread starting.
623------------------------------ Captured log setup ------------------------------
62416:46:15,392 conftest INFO Starting execution of test_cleanup at 2019-12-26 16:46:15.392402
62516:46:15,394 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-il3_gktm
626------------------------------ Captured log call -------------------------------
62716:46:31,813 cassandra.cluster INFO New Cassandra host <Host: 127.0.0.2 datacenter1> discovered
62816:47:01,857 cassandra.cluster WARNING Host 127.0.0.2 has been marked down
62916:47:02,888 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 2.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
63016:47:04,959 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 4.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
63116:47:08,970 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 8.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
63216:47:16,972 cassandra.pool WARNING Error attempting to reconnect to 127.0.0.2, scheduling retry in 16.0 seconds: [Errno 61] Tried connecting to [('127.0.0.2', 9042)]. Last error: Connection refused
633--------------------------- Captured stdout teardown ---------------------------
63416:46:14,106 ccm DEBUG Log-watching thread exiting.
635--------------------------- Captured stdout teardown ---------------------------
63616:46:14,106 ccm DEBUG Log-watching thread exiting.
637--------------------------- Captured stdout teardown ---------------------------
63816:47:17,560 ccm DEBUG Log-watching thread exiting.
639_____________________ TestOfflineTools.test_sstableverify ______________________
640
641self = <offline_tools_test.TestOfflineTools object at 0x11a1c2a20>
642
643    @since('2.2')
644    def test_sstableverify(self):
645        """
646            Generate sstables and test offline verification works correctly
647            Test on bad input: nonexistent keyspace and sstables
648            Test on potential situations: deleted sstables, corrupted sstables
649            """
650        cluster = self.cluster
651        cluster.populate(3).start(wait_for_binary_proto=True)
652        node1, node2, node3 = cluster.nodelist()
653    
654        # test on nonexistent keyspace
655        try:
656            (out, err, rc) = node1.run_sstableverify("keyspace1", "standard1")
657        except ToolError as e:
658            assert "Unknown keyspace/table keyspace1.standard1" in repr(e)
659            assert e.exit_status == 1, str(e.exit_status)
660    
661        # test on nonexistent sstables:
662        node1.stress(['write', 'n=100', 'no-warmup', '-schema', 'replication(factor=3)',
663                      '-rate', 'threads=8'])
664        (out, err, rc) = node1.run_sstableverify("keyspace1", "standard1")
665        assert rc == 0, str(rc)
666    
667        # Generate multiple sstables and test works properly in the simple case
668        node1.stress(['write', 'n=100K', 'no-warmup', '-schema', 'replication(factor=3)',
669                      '-rate', 'threads=8'])
670        node1.flush()
671        node1.stress(['write', 'n=100K', 'no-warmup', '-schema', 'replication(factor=3)',
672                      '-rate', 'threads=8'])
673        node1.flush()
674        cluster.stop()
675    
676        (out, error, rc) = node1.run_sstableverify("keyspace1", "standard1")
677    
678        assert rc == 0, str(rc)
679    
680        # STDOUT of the sstableverify command consists of multiple lines which may contain
681        # Java-normalized paths. To later compare these with Python-normalized paths, we
682        # map over each line of out and replace Java-normalized paths with Python equivalents.
683        outlines = [re.sub("(?<=path=').*(?=')",
684                                           lambda match: os.path.normcase(match.group(0)),
685                                           line) for line in out.splitlines()]
686    
687        # check output is correct for each sstable
688        sstables = self._get_final_sstables(node1, "keyspace1", "standard1")
689    
690        for sstable in sstables:
691            verified = False
692            hashcomputed = False
693            for line in outlines:
694                if sstable in line:
695                    if "Verifying BigTableReader" in line:
696                        verified = True
697                    elif "Checking computed hash of BigTableReader" in line:
698                        hashcomputed = True
699                    else:
700                        logger.debug(line)
701    
702            logger.debug(verified)
703            logger.debug(hashcomputed)
704            logger.debug(sstable)
705            assert verified and hashcomputed
706    
707        # now try intentionally corrupting an sstable to see if hash computed is different and error recognized
708        sstable1 = sstables[1]
709        with open(sstable1, 'rb') as f:
710            sstabledata = bytearray(f.read())
711        with open(sstable1, 'wb') as out:
712            position = random.randrange(0, len(sstabledata))
713            sstabledata[position] = (sstabledata[position] + 1) % 256
714            out.write(sstabledata)
715    
716        # use verbose to get some coverage on it
717        try:
718>           (out, error, rc) = node1.run_sstableverify("keyspace1", "standard1", options=['-v'])
719
720offline_tools_test.py:291: 
721_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
722
723self = <ccmlib.node.Node object at 0x117269cc0>, keyspace = 'keyspace1'
724cf = 'standard1', options = ['-v']
725
726    def run_sstableverify(self, keyspace, cf, options=None):
727        p = self.run_sstableverify_process(keyspace, cf, options=options)
728>       return handle_external_tool_process(p, "sstableverify on {} : {} with options: {}".format(keyspace, cf, options))
729
730../../dtest/src/ccm/ccmlib/node.py:1231: 
731_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
732
733process = <subprocess.Popen object at 0x119fdbc50>
734cmd_args = "sstableverify on keyspace1 : standard1 with options: ['-v']"
735
736    def handle_external_tool_process(process, cmd_args):
737        out, err = process.communicate()
738        if (out is not None) and isinstance(out, bytes):
739            out = out.decode()
740        if (err is not None) and isinstance(err, bytes):
741            err = err.decode()
742        rc = process.returncode
743    
744        if rc != 0:
745>           raise ToolError(cmd_args, rc, out, err)
746E           ccmlib.node.ToolError: Subprocess sstableverify on keyspace1 : standard1 with options: ['-v'] exited with non-zero status; exit status: 1; 
747E           stdout: Verifying BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data2/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-15-big-Data.db') (7.296MiB)
748E           Deserializing sstable metadata for BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data2/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-15-big-Data.db') 
749E           Checking computed hash of BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data2/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-15-big-Data.db') 
750E           Verifying BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data0/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-13-big-Data.db') (7.331MiB)
751E           Deserializing sstable metadata for BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data0/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-13-big-Data.db') 
752E           Checking computed hash of BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data0/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-13-big-Data.db') 
753E           Verifying BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db') (7.307MiB)
754E           Deserializing sstable metadata for BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db') 
755E           Checking computed hash of BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db') 
756E           Corrupted SSTable : /private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db
757E           ; 
758E           stderr: Error verifying BigTableReader(path='/private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db'): Corrupted: /private/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db
759
760../../dtest/src/ccm/ccmlib/node.py:2127: ToolError
761
762During handling of the above exception, another exception occurred:
763
764self = <offline_tools_test.TestOfflineTools object at 0x11a1c2a20>
765
766    @since('2.2')
767    def test_sstableverify(self):
768        """
769            Generate sstables and test offline verification works correctly
770            Test on bad input: nonexistent keyspace and sstables
771            Test on potential situations: deleted sstables, corrupted sstables
772            """
773        cluster = self.cluster
774        cluster.populate(3).start(wait_for_binary_proto=True)
775        node1, node2, node3 = cluster.nodelist()
776    
777        # test on nonexistent keyspace
778        try:
779            (out, err, rc) = node1.run_sstableverify("keyspace1", "standard1")
780        except ToolError as e:
781            assert "Unknown keyspace/table keyspace1.standard1" in repr(e)
782            assert e.exit_status == 1, str(e.exit_status)
783    
784        # test on nonexistent sstables:
785        node1.stress(['write', 'n=100', 'no-warmup', '-schema', 'replication(factor=3)',
786                      '-rate', 'threads=8'])
787        (out, err, rc) = node1.run_sstableverify("keyspace1", "standard1")
788        assert rc == 0, str(rc)
789    
790        # Generate multiple sstables and test works properly in the simple case
791        node1.stress(['write', 'n=100K', 'no-warmup', '-schema', 'replication(factor=3)',
792                      '-rate', 'threads=8'])
793        node1.flush()
794        node1.stress(['write', 'n=100K', 'no-warmup', '-schema', 'replication(factor=3)',
795                      '-rate', 'threads=8'])
796        node1.flush()
797        cluster.stop()
798    
799        (out, error, rc) = node1.run_sstableverify("keyspace1", "standard1")
800    
801        assert rc == 0, str(rc)
802    
803        # STDOUT of the sstableverify command consists of multiple lines which may contain
804        # Java-normalized paths. To later compare these with Python-normalized paths, we
805        # map over each line of out and replace Java-normalized paths with Python equivalents.
806        outlines = [re.sub("(?<=path=').*(?=')",
807                                           lambda match: os.path.normcase(match.group(0)),
808                                           line) for line in out.splitlines()]
809    
810        # check output is correct for each sstable
811        sstables = self._get_final_sstables(node1, "keyspace1", "standard1")
812    
813        for sstable in sstables:
814            verified = False
815            hashcomputed = False
816            for line in outlines:
817                if sstable in line:
818                    if "Verifying BigTableReader" in line:
819                        verified = True
820                    elif "Checking computed hash of BigTableReader" in line:
821                        hashcomputed = True
822                    else:
823                        logger.debug(line)
824    
825            logger.debug(verified)
826            logger.debug(hashcomputed)
827            logger.debug(sstable)
828            assert verified and hashcomputed
829    
830        # now try intentionally corrupting an sstable to see if hash computed is different and error recognized
831        sstable1 = sstables[1]
832        with open(sstable1, 'rb') as f:
833            sstabledata = bytearray(f.read())
834        with open(sstable1, 'wb') as out:
835            position = random.randrange(0, len(sstabledata))
836            sstabledata[position] = (sstabledata[position] + 1) % 256
837            out.write(sstabledata)
838    
839        # use verbose to get some coverage on it
840        try:
841            (out, error, rc) = node1.run_sstableverify("keyspace1", "standard1", options=['-v'])
842        except ToolError as e:
843            # Process sstableverify output to normalize paths in string to Python casing as above
844            error = re.sub("(?<=Corrupted: ).*", lambda match: os.path.normcase(match.group(0)), str(e))
845    
846>           assert re.search("Corrupted: " + sstable1, error)
847E           assert None
848E            +  where None = <function search at 0x106e461e0>(('Corrupted: ' + '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db'), "Subprocess sstableverify on keyspace1 : standard1 with options: ['-v'] exited with non-zero status; exit status: 1; \...nhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db\n")
849E            +    where <function search at 0x106e461e0> = re.search
850
851offline_tools_test.py:296: AssertionError
852---------------------------- Captured stdout setup -----------------------------
85317:39:38,610 ccm DEBUG Log-watching thread starting.
854------------------------------ Captured log setup ------------------------------
85517:39:38,536 conftest INFO Starting execution of test_sstableverify at 2019-12-26 17:39:38.536936
85617:39:38,537 dtest_setup INFO cluster ccm directory: /var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a
857--------------------------- Captured stdout teardown ---------------------------
85817:41:26,868 ccm DEBUG Log-watching thread exiting.
859===Flaky Test Report===
860
861test_login_existing_node passed 1 out of the required 1 times. Success!
862test_login_new_node passed 1 out of the required 1 times. Success!
863test_list_users passed 1 out of the required 1 times. Success!
864test_modify_and_select_auth passed 1 out of the required 1 times. Success!
865test_system_auth_ks_is_alterable passed 1 out of the required 1 times. Success!
866test_login passed 1 out of the required 1 times. Success!
867test_cant_create_existing_user passed 1 out of the required 1 times. Success!
868test_list_users passed 1 out of the required 1 times. Success!
869test_handle_corrupt_role_data passed 1 out of the required 1 times. Success!
870test_user_cant_drop_themselves passed 1 out of the required 1 times. Success!
871test_dropping_nonexistent_user_throws_exception passed 1 out of the required 1 times. Success!
872test_drop_user_case_sensitive passed 1 out of the required 1 times. Success!
873test_alter_user_case_sensitive passed 1 out of the required 1 times. Success!
874test_regular_users_can_alter_their_passwords_only passed 1 out of the required 1 times. Success!
875test_users_cant_alter_their_superuser_status passed 1 out of the required 1 times. Success!
876test_only_superuser_alters_superuser_status passed 1 out of the required 1 times. Success!
877test_altering_nonexistent_user_throws_exception passed 1 out of the required 1 times. Success!
878test_conditional_create_drop_user passed 1 out of the required 1 times. Success!
879test_create_ks_auth passed 1 out of the required 1 times. Success!
880test_create_cf_auth passed 1 out of the required 1 times. Success!
881test_alter_ks_auth passed 1 out of the required 1 times. Success!
882test_alter_cf_auth passed 1 out of the required 1 times. Success!
883test_materialized_views_auth passed 1 out of the required 1 times. Success!
884test_drop_ks_auth passed 1 out of the required 1 times. Success!
885test_drop_cf_auth passed 1 out of the required 1 times. Success!
886test_modify_and_select_auth passed 1 out of the required 1 times. Success!
887test_grant_revoke_without_ks_specified passed 1 out of the required 1 times. Success!
888test_grant_revoke_auth passed 1 out of the required 1 times. Success!
889test_grant_revoke_nonexistent_user_or_ks passed 1 out of the required 1 times. Success!
890test_grant_revoke_cleanup passed 1 out of the required 1 times. Success!
891test_permissions_caching passed 1 out of the required 1 times. Success!
892test_list_permissions passed 1 out of the required 1 times. Success!
893test_type_auth passed 1 out of the required 1 times. Success!
894test_restart_node_doesnt_lose_auth_data passed 1 out of the required 1 times. Success!
895test_auth_metrics passed 1 out of the required 1 times. Success!
896test_create_drop_role passed 1 out of the required 1 times. Success!
897test_conditional_create_drop_role passed 1 out of the required 1 times. Success!
898test_create_drop_role_validation passed 1 out of the required 1 times. Success!
899test_role_admin_validation passed 1 out of the required 1 times. Success!
900test_creator_of_db_resource_granted_all_permissions passed 1 out of the required 1 times. Success!
901test_create_and_grant_roles_with_superuser_status passed 1 out of the required 1 times. Success!
902test_drop_and_revoke_roles_with_superuser_status passed 1 out of the required 1 times. Success!
903test_drop_role_removes_memberships passed 1 out of the required 1 times. Success!
904test_drop_role_revokes_permissions_granted_on_it passed 1 out of the required 1 times. Success!
905test_grant_revoke_roles passed 1 out of the required 1 times. Success!
906test_grant_revoke_role_validation passed 1 out of the required 1 times. Success!
907test_list_roles passed 1 out of the required 1 times. Success!
908test_grant_revoke_permissions passed 1 out of the required 1 times. Success!
909test_filter_granted_permissions_by_resource_type passed 1 out of the required 1 times. Success!
910test_list_permissions passed 1 out of the required 1 times. Success!
911test_list_permissions_validation passed 1 out of the required 1 times. Success!
912test_role_caching_authenticated_user passed 1 out of the required 1 times. Success!
913test_drop_non_existent_role_should_not_update_cache passed 1 out of the required 1 times. Success!
914test_prevent_circular_grants passed 1 out of the required 1 times. Success!
915test_create_user_as_alias_for_create_role passed 1 out of the required 1 times. Success!
916test_role_name passed 1 out of the required 1 times. Success!
917test_role_requires_login_privilege_to_authenticate passed 1 out of the required 1 times. Success!
918test_roles_do_not_inherit_login_privilege passed 1 out of the required 1 times. Success!
919test_role_requires_password_to_login passed 1 out of the required 1 times. Success!
920test_superuser_status_is_inherited passed 1 out of the required 1 times. Success!
921test_list_users_considers_inherited_superuser_status passed 1 out of the required 1 times. Success!
922test_grant_revoke_udf_permissions passed 1 out of the required 1 times. Success!
923test_grant_revoke_are_idempotent passed 1 out of the required 1 times. Success!
924test_function_resource_hierarchy_permissions passed 1 out of the required 1 times. Success!
925test_udf_permissions_validation passed 1 out of the required 1 times. Success!
926test_drop_role_cleans_up_udf_permissions passed 1 out of the required 1 times. Success!
927test_drop_function_and_keyspace_cleans_up_udf_permissions passed 1 out of the required 1 times. Success!
928test_udf_with_overloads_permissions passed 1 out of the required 1 times. Success!
929test_drop_keyspace_cleans_up_function_level_permissions passed 1 out of the required 1 times. Success!
930test_udf_permissions_in_selection passed 1 out of the required 1 times. Success!
931test_udf_permissions_in_select_where_clause passed 1 out of the required 1 times. Success!
932test_udf_permissions_in_insert passed 1 out of the required 1 times. Success!
933test_udf_permissions_in_update passed 1 out of the required 1 times. Success!
934test_udf_permissions_in_delete passed 1 out of the required 1 times. Success!
935test_inheritence_of_udf_permissions passed 1 out of the required 1 times. Success!
936test_builtin_functions_require_no_special_permissions passed 1 out of the required 1 times. Success!
937test_disallow_grant_revoke_on_builtin_functions passed 1 out of the required 1 times. Success!
938test_disallow_grant_execute_on_non_function_resources passed 1 out of the required 1 times. Success!
939test_aggregate_function_permissions passed 1 out of the required 1 times. Success!
940test_ignore_invalid_roles passed 1 out of the required 1 times. Success!
941test_authentication_handle_unavailable passed 1 out of the required 1 times. Success!
942test_authentication_through_cache_handle_unavailable passed 1 out of the required 1 times. Success!
943test_authorization_handle_unavailable passed 1 out of the required 1 times. Success!
944test_authorization_through_cache_handle_unavailable passed 1 out of the required 1 times. Success!
945test_authorization_from_cache_while_unavailable passed 1 out of the required 1 times. Success!
946test_permission_cache_background_reload_handle_unavailable passed 1 out of the required 1 times. Success!
947test_empty_batch_throws_no_error passed 1 out of the required 1 times. Success!
948test_counter_batch_accepts_counter_mutations passed 1 out of the required 1 times. Success!
949test_counter_batch_rejects_regular_mutations passed 1 out of the required 1 times. Success!
950test_logged_batch_accepts_regular_mutations passed 1 out of the required 1 times. Success!
951test_logged_batch_gcgs_below_threshold_single_table passed 1 out of the required 1 times. Success!
952test_logged_batch_gcgs_below_threshold_multi_table passed 1 out of the required 1 times. Success!
953test_unlogged_batch_gcgs_below_threshold_should_not_print_warning passed 1 out of the required 1 times. Success!
954test_logged_batch_rejects_counter_mutations passed 1 out of the required 1 times. Success!
955test_unlogged_batch_accepts_regular_mutations passed 1 out of the required 1 times. Success!
956test_unlogged_batch_rejects_counter_mutations passed 1 out of the required 1 times. Success!
957test_logged_batch_throws_uae passed 1 out of the required 1 times. Success!
958test_logged_batch_doesnt_throw_uae passed 1 out of the required 1 times. Success!
959test_acknowledged_by_batchlog_not_set_when_batchlog_write_fails passed 1 out of the required 1 times. Success!
960test_acknowledged_by_batchlog_set_when_batchlog_write_succeeds passed 1 out of the required 1 times. Success!
961test_batch_uses_proper_timestamp passed 1 out of the required 1 times. Success!
962test_only_one_timestamp_is_valid passed 1 out of the required 1 times. Success!
963test_each_statement_in_batch_uses_proper_timestamp passed 1 out of the required 1 times. Success!
964test_multi_table_batch_for_10554 passed 1 out of the required 1 times. Success!
965test_logged_batch_compatibility_1 passed 1 out of the required 1 times. Success!
966test_batchlog_replay_compatibility_1 passed 1 out of the required 1 times. Success!
967test_logged_batch_compatibility_2 passed 1 out of the required 1 times. Success!
968test_logged_batch_compatibility_3 passed 1 out of the required 1 times. Success!
969test_logged_batch_compatibility_4 passed 1 out of the required 1 times. Success!
970test_batchlog_replay_compatibility_4 passed 1 out of the required 1 times. Success!
971test_logged_batch_compatibility_5 passed 1 out of the required 1 times. Success!
972test_simple_bootstrap_with_ssl passed 1 out of the required 1 times. Success!
973test_simple_bootstrap passed 1 out of the required 1 times. Success!
974test_bootstrap_on_write_survey passed 1 out of the required 1 times. Success!
975test_simple_bootstrap_small_keepalive_period passed 1 out of the required 1 times. Success!
976test_simple_bootstrap_nodata passed 1 out of the required 1 times. Success!
977test_read_from_bootstrapped_node passed 1 out of the required 1 times. Success!
978test_consistent_range_movement_true_with_replica_down_should_fail passed 1 out of the required 1 times. Success!
979test_consistent_range_movement_false_with_replica_down_should_succeed passed 1 out of the required 1 times. Success!
980test_consistent_range_movement_true_with_rf1_should_fail passed 1 out of the required 1 times. Success!
981test_consistent_range_movement_false_with_rf1_should_succeed passed 1 out of the required 1 times. Success!
982test_resumable_bootstrap passed 1 out of the required 1 times. Success!
983test_bootstrap_with_reset_bootstrap_state passed 1 out of the required 1 times. Success!
984test_manual_bootstrap passed 1 out of the required 1 times. Success!
985test_local_quorum_bootstrap passed 1 out of the required 1 times. Success!
986test_shutdown_wiped_node_cannot_join passed 1 out of the required 1 times. Success!
987test_killed_wiped_node_cannot_join passed 1 out of the required 1 times. Success!
988test_decommissioned_wiped_node_can_join passed 1 out of the required 1 times. Success!
989test_decommissioned_wiped_node_can_gossip_to_single_seed passed 1 out of the required 1 times. Success!
990test_failed_bootstrap_wiped_node_can_join passed 1 out of the required 1 times. Success!
991test_simultaneous_bootstrap passed 1 out of the required 1 times. Success!
992test_cleanup passed 1 out of the required 1 times. Success!
993test_bootstrap_binary_disabled failed and was not selected for rerun.
994	<class 'Failed'>
995	Log message was not seen within timeout:
996Not starting client transports as bootstrap has not completed
997	[<TracebackEntry /Users/asp/development/cassandra-dtest/bootstrap_test.py:763>, <TracebackEntry /Users/asp/development/cassandra-dtest/dtest.py:271>]
998test_cdc_enabled_data_readable_on_round_trip passed 1 out of the required 1 times. Success!
999test_cdc_disabled_data_readable_on_round_trip passed 1 out of the required 1 times. Success!
1000test_non_cdc_segments_deleted_after_replay passed 1 out of the required 1 times. Success!
1001test_insertion_and_commitlog_behavior_after_reaching_cdc_total_space passed 1 out of the required 1 times. Success!
1002test_cdc_data_available_in_cdc_raw passed 1 out of the required 1 times. Success!
1003test_cfid passed 1 out of the required 1 times. Success!
1004test_mv_lock_contention_during_replay passed 1 out of the required 1 times. Success!
1005test_commitlog_replay_on_startup passed 1 out of the required 1 times. Success!
1006test_default_segment_size passed 1 out of the required 1 times. Success!
1007test_small_segment_size passed 1 out of the required 1 times. Success!
1008test_default_compressed_segment_size passed 1 out of the required 1 times. Success!
1009test_small_compressed_segment_size passed 1 out of the required 1 times. Success!
1010test_stop_failure_policy passed 1 out of the required 1 times. Success!
1011test_stop_commit_failure_policy passed 1 out of the required 1 times. Success!
1012test_die_failure_policy passed 1 out of the required 1 times. Success!
1013test_ignore_failure_policy passed 1 out of the required 1 times. Success!
1014test_bad_crc passed 1 out of the required 1 times. Success!
1015test_compression_error passed 1 out of the required 1 times. Success!
1016test_data_size passed 1 out of the required 1 times. Success!
1017test_bloomfilter_size[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1018test_bloomfilter_size[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1019test_bloomfilter_size[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1020test_sstable_deletion[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1021test_sstable_deletion[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1022test_sstable_deletion[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1023test_dtcs_deletion[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1024test_compaction_throughput passed 1 out of the required 1 times. Success!
1025test_compaction_strategy_switching[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1026test_compaction_strategy_switching[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1027test_compaction_strategy_switching[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1028test_large_compaction_warning passed 1 out of the required 1 times. Success!
1029test_disable_autocompaction_nodetool[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1030test_disable_autocompaction_nodetool[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1031test_disable_autocompaction_nodetool[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1032test_disable_autocompaction_schema[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1033test_disable_autocompaction_schema[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1034test_disable_autocompaction_schema[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1035test_disable_autocompaction_alter[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1036test_disable_autocompaction_alter[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1037test_disable_autocompaction_alter[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1038test_disable_autocompaction_alter_and_nodetool[LeveledCompactionStrategy] passed 1 out of the required 1 times. Success!
1039test_disable_autocompaction_alter_and_nodetool[SizeTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1040test_disable_autocompaction_alter_and_nodetool[DateTieredCompactionStrategy] passed 1 out of the required 1 times. Success!
1041test_user_defined_compaction passed 1 out of the required 1 times. Success!
1042test_disable_compression_cql passed 1 out of the required 1 times. Success!
1043test_compression_cql_options passed 1 out of the required 1 times. Success!
1044test_compression_cql_disabled_with_alter passed 1 out of the required 1 times. Success!
1045test_compression_cql_enabled_with_alter passed 1 out of the required 1 times. Success!
1046test_compression_chunk_length passed 1 out of the required 1 times. Success!
1047test_change_durable_writes passed 1 out of the required 1 times. Success!
1048test_simple_strategy passed 1 out of the required 1 times. Success!
1049test_simple_strategy_each_quorum passed 1 out of the required 1 times. Success!
1050test_simple_strategy_counters passed 1 out of the required 1 times. Success!
1051test_simple_strategy_each_quorum_counters passed 1 out of the required 1 times. Success!
1052test_14513_transient passed 1 out of the required 1 times. Success!
1053test_14513_permanent passed 1 out of the required 1 times. Success!
1054test_14330 passed 1 out of the required 1 times. Success!
1055test_13911 passed 1 out of the required 1 times. Success!
1056test_13911_rows_srp passed 1 out of the required 1 times. Success!
1057test_13911_partitions_srp passed 1 out of the required 1 times. Success!
1058test_13880 passed 1 out of the required 1 times. Success!
1059test_13747 passed 1 out of the required 1 times. Success!
1060test_13595 passed 1 out of the required 1 times. Success!
1061test_12872 passed 1 out of the required 1 times. Success!
1062test_short_read failed and was not selected for rerun.
1063	<class 'OSError'>
1064	[Errno 24] Too many open files: '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-4mfru3_j/test/node1/bin/cassandra.in.sh'
1065	[<TracebackEntry /Users/asp/development/cassandra-dtest/consistency_test.py:1395>, <TracebackEntry /Users/asp/development/cassandra-dtest/consistency_test.py:1576>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:1375>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:853>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:845>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:205>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/common.py:280>, <TracebackEntry /Users/asp/dtest/lib/python3.7/shutil.py:241>, <TracebackEntry /Users/asp/dtest/lib/python3.7/shutil.py:121>]
1066test_short_read_delete passed 1 out of the required 1 times. Success!
1067test_short_read_quorum_delete passed 1 out of the required 1 times. Success!
1068test_readrepair passed 1 out of the required 1 times. Success!
1069test_quorum_available_during_failure passed 1 out of the required 1 times. Success!
1070test_consistent_reads_after_move passed 1 out of the required 1 times. Success!
1071test_consistent_reads_after_bootstrap passed 1 out of the required 1 times. Success!
1072test_13691 passed 1 out of the required 1 times. Success!
1073test_simple_increment passed 1 out of the required 1 times. Success!
1074test_upgrade passed 1 out of the required 1 times. Success!
1075test_counter_consistency passed 1 out of the required 1 times. Success!
1076test_multi_counter_update passed 1 out of the required 1 times. Success!
1077test_validate_empty_column_name passed 1 out of the required 1 times. Success!
1078test_drop_counter_column passed 1 out of the required 1 times. Success!
1079test_compact_counter_cluster passed 1 out of the required 1 times. Success!
1080test_batch_preparation passed 1 out of the required 1 times. Success!
1081test_keyspace passed 1 out of the required 1 times. Success!
1082test_table passed 1 out of the required 1 times. Success!
1083test_table_compact_storage passed 1 out of the required 1 times. Success!
1084test_index passed 1 out of the required 1 times. Success!
1085test_type passed 1 out of the required 1 times. Success!
1086test_user passed 1 out of the required 1 times. Success!
1087test_statements passed 1 out of the required 1 times. Success!
1088test_partition_key_allow_filtering passed 1 out of the required 1 times. Success!
1089test_batch passed 1 out of the required 1 times. Success!
1090test_cql3_insert_thrift failed and was not selected for rerun.
1091	<class 'thrift.transport.TTransport.TTransportException'>
1092	Could not connect to any of [('127.0.0.1', 9160)]
1093	[<TracebackEntry /Users/asp/development/cassandra-dtest/cql_test.py:547>, <TracebackEntry /Users/asp/dtest/lib/python3.7/site-packages/thrift/transport/TTransport.py:269>, <TracebackEntry /Users/asp/dtest/lib/python3.7/site-packages/thrift/transport/TSocket.py:113>]
1094test_rename passed 1 out of the required 1 times. Success!
1095test_invalid_string_literals passed 1 out of the required 1 times. Success!
1096test_prepared_statement_invalidation passed 1 out of the required 1 times. Success!
1097test_range_slice passed 1 out of the required 1 times. Success!
1098test_drop_compact_storage_flag passed 1 out of the required 1 times. Success!
1099test_local_query passed 1 out of the required 1 times. Success!
1100test_remote_query passed 1 out of the required 1 times. Success!
1101test_disable_slow_query_log passed 1 out of the required 1 times. Success!
1102test_lwt_with_static_columns passed 1 out of the required 1 times. Success!
1103test_conditional_updates_on_static_columns_with_null_values passed 1 out of the required 1 times. Success!
1104test_conditional_updates_on_static_columns_with_non_existing_values passed 1 out of the required 1 times. Success!
1105test_conditional_updates_on_static_columns_with_null_values_batch passed 1 out of the required 1 times. Success!
1106test_conditional_deletes_on_static_columns_with_null_values passed 1 out of the required 1 times. Success!
1107test_conditional_deletes_on_static_columns_with_null_values_batch passed 1 out of the required 1 times. Success!
1108test_tracing_simple passed 1 out of the required 1 times. Success!
1109test_tracing_unknown_impl passed 1 out of the required 1 times. Success!
1110test_tracing_default_impl passed 1 out of the required 1 times. Success!
1111test_tracing_does_not_interfere_with_digest_calculation passed 1 out of the required 1 times. Success!
1112test_delete_insert_search passed 1 out of the required 1 times. Success!
1113test_gc passed 1 out of the required 1 times. Success!
1114test_tombstone_size passed 1 out of the required 1 times. Success!
1115test_disk_balance_stress passed 1 out of the required 1 times. Success!
1116test_disk_balance_replace_same_address passed 1 out of the required 1 times. Success!
1117test_disk_balance_replace_different_address passed 1 out of the required 1 times. Success!
1118test_disk_balance_decommission passed 1 out of the required 1 times. Success!
1119test_blacklisted_directory passed 1 out of the required 1 times. Success!
1120test_alter_replication_factor passed 1 out of the required 1 times. Success!
1121test_disk_balance_after_boundary_change_stcs passed 1 out of the required 1 times. Success!
1122test_disk_balance_after_boundary_change_lcs passed 1 out of the required 1 times. Success!
1123test_disk_balance_after_joining_ring_stcs passed 1 out of the required 1 times. Success!
1124test_disk_balance_after_joining_ring_lcs passed 1 out of the required 1 times. Success!
1125test_functional passed 1 out of the required 1 times. Success!
1126test_nodetool passed 1 out of the required 1 times. Success!
1127test_hintedhandoff_disabled passed 1 out of the required 1 times. Success!
1128test_hintedhandoff_enabled passed 1 out of the required 1 times. Success!
1129test_hintedhandoff_dc_disabled passed 1 out of the required 1 times. Success!
1130test_hintedhandoff_dc_reenabled passed 1 out of the required 1 times. Success!
1131test_hintedhandoff_decom passed 1 out of the required 1 times. Success!
1132test_putget_with_internode_ssl passed 1 out of the required 1 times. Success!
1133test_putget_with_internode_ssl_without_compression passed 1 out of the required 1 times. Success!
1134test_basic_auth passed 1 out of the required 1 times. Success!
1135test_netstats passed 1 out of the required 1 times. Success!
1136test_table_metric_mbeans passed 1 out of the required 1 times. Success!
1137test_mv_metric_mbeans_release passed 1 out of the required 1 times. Success!
1138test_compactionstats passed 1 out of the required 1 times. Success!
1139test_phi passed 1 out of the required 1 times. Success!
1140test_jmx_connection passed 1 out of the required 1 times. Success!
1141test_require_client_auth passed 1 out of the required 1 times. Success!
1142test_basic_data_types passed 1 out of the required 1 times. Success!
1143test_counters passed 1 out of the required 1 times. Success!
1144test_complex_data_types passed 1 out of the required 1 times. Success!
1145test_basic_data_types passed 1 out of the required 1 times. Success!
1146test_complex_data_types passed 1 out of the required 1 times. Success!
1147test_collection_update passed 1 out of the required 1 times. Success!
1148test_selecting_pkey_as_json passed 1 out of the required 1 times. Success!
1149test_select_using_secondary_index passed 1 out of the required 1 times. Success!
1150test_basic_data_types passed 1 out of the required 1 times. Success!
1151test_complex_data_types passed 1 out of the required 1 times. Success!
1152test_delete_using_pkey_json passed 1 out of the required 1 times. Success!
1153test_simple_schema passed 1 out of the required 1 times. Success!
1154test_pkey_requirement passed 1 out of the required 1 times. Success!
1155test_null_value passed 1 out of the required 1 times. Success!
1156test_complex_schema passed 1 out of the required 1 times. Success!
1157test_cleanup failed (1 runs remaining out of 2).
1158	<class 'ccmlib.node.ToolError'>
1159	Subprocess ['stress', 'write', 'n=5', 'no-warmup', 'cl=ALL', '-pop', 'seq=1...5', '-schema', 'replication(factor=2)', '-col', 'n=fixed(1)', 'size=fixed(66060288)', '-rate', 'threads=1'] exited with non-zero status; exit status: 1; 
1160stdout: ******************** Stress Settings ********************
1161Command:
1162  Type: write
1163  Count: 5
1164  No Warmup: true
1165  Consistency Level: ALL
1166  Target Uncertainty: not applicable
1167  Key Size (bytes): 10
1168  Counter Increment Distibution: add=fixed(1)
1169Rate:
1170  Auto: false
1171  Thread Count: 1
1172  OpsPer Sec: 0
1173Population:
1174  Sequence: 1..5
1175  Order: ARBITRARY
1176  Wrap: true
1177Insert:
1178  Revisits: Uniform:  min=1,max=1000000
1179  Visits: Fixed:  key=1
1180  Row Population Ratio: Ratio: divisor=1.000000;delegate=Fixed:  key=1
1181  Batch Type: not batching
1182Columns:
1183  Max Columns Per Key: 1
1184  Column Names: [C0]
1185  Comparator: AsciiType
1186  Timestamp: null
1187  Variable Column Count: false
1188  Slice: false
1189  Size Distribution: Fixed:  key=66060288
1190  Count Distribution: Fixed:  key=1
1191Errors:
1192  Ignore: false
1193  Tries: 10
1194Log:
1195  No Summary: false
1196  No Settings: false
1197  File: null
1198  Interval Millis: 1000
1199  Level: NORMAL
1200Mode:
1201  API: JAVA_DRIVER_NATIVE
1202  Connection Style: CQL_PREPARED
1203  CQL Version: CQL3
1204  Protocol Version: V4
1205  Username: null
1206  Password: null
1207  Auth Provide Class: null
1208  Max Pending Per Connection: 128
1209  Connections Per Host: 8
1210  Compression: NONE
1211Node:
1212  Nodes: [127.0.0.1]
1213  Is White List: false
1214  Datacenter: null
1215Schema:
1216  Keyspace: keyspace1
1217  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
1218  Replication Strategy Pptions: {replication_factor=2}
1219  Table Compression: null
1220  Table Compaction Strategy: null
1221  Table Compaction Strategy Options: {}
1222Transport:
1223  factory=org.apache.cassandra.thrift.TFramedTransportFactory; truststore=null; truststore-password=null; keystore=null; keystore-password=null; ssl-protocol=TLS; ssl-alg=SunX509; store-type=JKS; ssl-ciphers=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA; 
1224Port:
1225  Native Port: 9042
1226  Thrift Port: 9160
1227  JMX Port: 7100
1228Send To Daemon:
1229  *not set*
1230Graph:
1231  File: null
1232  Revision: unknown
1233  Title: null
1234  Operation: WRITE
1235TokenRange:
1236  Wrap: false
1237  Split Factor: 1
1238
1239Connected to cluster: test, max pending requests per connection 128, max connections per host 8
1240Datatacenter: datacenter1; Host: /127.0.0.1; Rack: rack1
1241Datatacenter: datacenter1; Host: /127.0.0.2; Rack: rack1
1242Created keyspaces. Sleeping 1s for propagation.
1243Sleeping 2s...
1244Running WRITE with 1 threads for 5 iteration
1245type       total ops,    op/s,    pk/s,   row/s,    mean,     med,     .95,     .99,    .999,     max,   time,   stderr, errors,  gc: #,  max ms,  sum ms,  sdv ms,      mb
1246java.io.IOException: Operation x10 on key(s) [30503337373039503231]: Error executing: (NoHostAvailableException): All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1247
1248	at org.apache.cassandra.stress.Operation.error(Operation.java:136)
1249	at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:114)
1250	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
1251	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
1252	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:264)
1253	at org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:472)
1254total,             1,      16,      16,      16, 17540.6, 17549.0, 17549.0, 17549.0, 17549.0, 17549.0,   18.1, -0.97333,      1,      0,       0,       0,       0,       0
1255FAILURE
1256; 
1257stderr: Failed to connect over JMX; not collecting these stats
1258com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.OperationTimedOutException: [localhost/127.0.0.1] Timed out waiting for server response), 127.0.0.2/127.0.0.2:9042 (com.datastax.driver.core.exceptions.TransportException: [127.0.0.2/127.0.0.2] Connection has been closed))
1259com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1260com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1261com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1262com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1263com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1264com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1265com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1266com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1267com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1268java.lang.RuntimeException: Failed to execute stress action
1269	at org.apache.cassandra.stress.StressAction.run(StressAction.java:100)
1270	at org.apache.cassandra.stress.Stress.run(Stress.java:143)
1271	at org.apache.cassandra.stress.Stress.main(Stress.java:62)
1272
1273	[<TracebackEntry /Users/asp/development/cassandra-dtest/largecolumn_test.py:67>, <TracebackEntry /Users/asp/development/cassandra-dtest/largecolumn_test.py:20>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:1348>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:2127>]
1274test_cleanup failed; it passed 0 out of the required 1 times.
1275	<class 'ccmlib.node.ToolError'>
1276	Subprocess ['stress', 'write', 'n=5', 'no-warmup', 'cl=ALL', '-pop', 'seq=1...5', '-schema', 'replication(factor=2)', '-col', 'n=fixed(1)', 'size=fixed(66060288)', '-rate', 'threads=1'] exited with non-zero status; exit status: 1; 
1277stdout: ******************** Stress Settings ********************
1278Command:
1279  Type: write
1280  Count: 5
1281  No Warmup: true
1282  Consistency Level: ALL
1283  Target Uncertainty: not applicable
1284  Key Size (bytes): 10
1285  Counter Increment Distibution: add=fixed(1)
1286Rate:
1287  Auto: false
1288  Thread Count: 1
1289  OpsPer Sec: 0
1290Population:
1291  Sequence: 1..5
1292  Order: ARBITRARY
1293  Wrap: true
1294Insert:
1295  Revisits: Uniform:  min=1,max=1000000
1296  Visits: Fixed:  key=1
1297  Row Population Ratio: Ratio: divisor=1.000000;delegate=Fixed:  key=1
1298  Batch Type: not batching
1299Columns:
1300  Max Columns Per Key: 1
1301  Column Names: [C0]
1302  Comparator: AsciiType
1303  Timestamp: null
1304  Variable Column Count: false
1305  Slice: false
1306  Size Distribution: Fixed:  key=66060288
1307  Count Distribution: Fixed:  key=1
1308Errors:
1309  Ignore: false
1310  Tries: 10
1311Log:
1312  No Summary: false
1313  No Settings: false
1314  File: null
1315  Interval Millis: 1000
1316  Level: NORMAL
1317Mode:
1318  API: JAVA_DRIVER_NATIVE
1319  Connection Style: CQL_PREPARED
1320  CQL Version: CQL3
1321  Protocol Version: V4
1322  Username: null
1323  Password: null
1324  Auth Provide Class: null
1325  Max Pending Per Connection: 128
1326  Connections Per Host: 8
1327  Compression: NONE
1328Node:
1329  Nodes: [127.0.0.1]
1330  Is White List: false
1331  Datacenter: null
1332Schema:
1333  Keyspace: keyspace1
1334  Replication Strategy: org.apache.cassandra.locator.SimpleStrategy
1335  Replication Strategy Pptions: {replication_factor=2}
1336  Table Compression: null
1337  Table Compaction Strategy: null
1338  Table Compaction Strategy Options: {}
1339Transport:
1340  factory=org.apache.cassandra.thrift.TFramedTransportFactory; truststore=null; truststore-password=null; keystore=null; keystore-password=null; ssl-protocol=TLS; ssl-alg=SunX509; store-type=JKS; ssl-ciphers=TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA; 
1341Port:
1342  Native Port: 9042
1343  Thrift Port: 9160
1344  JMX Port: 7100
1345Send To Daemon:
1346  *not set*
1347Graph:
1348  File: null
1349  Revision: unknown
1350  Title: null
1351  Operation: WRITE
1352TokenRange:
1353  Wrap: false
1354  Split Factor: 1
1355
1356Connected to cluster: test, max pending requests per connection 128, max connections per host 8
1357Datatacenter: datacenter1; Host: /127.0.0.1; Rack: rack1
1358Datatacenter: datacenter1; Host: /127.0.0.2; Rack: rack1
1359Created keyspaces. Sleeping 1s for propagation.
1360Sleeping 2s...
1361Running WRITE with 1 threads for 5 iteration
1362type       total ops,    op/s,    pk/s,   row/s,    mean,     med,     .95,     .99,    .999,     max,   time,   stderr, errors,  gc: #,  max ms,  sum ms,  sdv ms,      mb
1363java.io.IOException: Operation x10 on key(s) [30503337373039503231]: Error executing: (NoHostAvailableException): All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1364
1365	at org.apache.cassandra.stress.Operation.error(Operation.java:136)
1366	at org.apache.cassandra.stress.Operation.timeWithRetry(Operation.java:114)
1367	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:101)
1368	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:109)
1369	at org.apache.cassandra.stress.operations.predefined.CqlOperation.run(CqlOperation.java:264)
1370	at org.apache.cassandra.stress.StressAction$Consumer.run(StressAction.java:472)
1371total,             1,       1,       1,       1, 17758.7, 17767.1, 17767.1, 17767.1, 17767.1, 17767.1,   18.8, -0.97333,      1,      0,       0,       0,       0,       0
1372FAILURE
1373; 
1374stderr: Failed to connect over JMX; not collecting these stats
1375com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.OperationTimedOutException: [localhost/127.0.0.1] Timed out waiting for server response), 127.0.0.2/127.0.0.2:9042 (com.datastax.driver.core.exceptions.TransportException: [127.0.0.2/127.0.0.2] Connection has been closed))
1376com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1377com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1378com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1379com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1380com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1381com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1382com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1383com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1384com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.UnavailableException: Not enough replicas available for query at consistency ALL (2 required but only 1 alive)))
1385java.lang.RuntimeException: Failed to execute stress action
1386	at org.apache.cassandra.stress.StressAction.run(StressAction.java:100)
1387	at org.apache.cassandra.stress.Stress.run(Stress.java:143)
1388	at org.apache.cassandra.stress.Stress.main(Stress.java:62)
1389
1390	[<TracebackEntry /Users/asp/development/cassandra-dtest/largecolumn_test.py:67>, <TracebackEntry /Users/asp/development/cassandra-dtest/largecolumn_test.py:20>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:1348>, <TracebackEntry /Users/asp/dtest/src/ccm/ccmlib/node.py:2127>]
1391test_14766 passed 1 out of the required 1 times. Success!
1392test_view_metadata_cleanup passed 1 out of the required 1 times. Success!
1393test_create passed 1 out of the required 1 times. Success!
1394test_gcgs_validation passed 1 out of the required 1 times. Success!
1395test_insert passed 1 out of the required 1 times. Success!
1396test_populate_mv_after_insert passed 1 out of the required 1 times. Success!
1397test_populate_mv_after_insert_wide_rows passed 1 out of the required 1 times. Success!
1398test_crc_check_chance passed 1 out of the required 1 times. Success!
1399test_prepared_statement passed 1 out of the required 1 times. Success!
1400test_immutable passed 1 out of the required 1 times. Success!
1401test_drop_mv passed 1 out of the required 1 times. Success!
1402test_drop_column passed 1 out of the required 1 times. Success!
1403test_drop_table passed 1 out of the required 1 times. Success!
1404test_clustering_column passed 1 out of the required 1 times. Success!
1405test_insert_during_range_movement_rf1 passed 1 out of the required 1 times. Success!
1406test_insert_during_range_movement_rf2 passed 1 out of the required 1 times. Success!
1407test_insert_during_range_movement_rf3 passed 1 out of the required 1 times. Success!
1408test_allow_filtering passed 1 out of the required 1 times. Success!
1409test_secondary_index passed 1 out of the required 1 times. Success!
1410test_ttl passed 1 out of the required 1 times. Success!
1411test_query_all_new_column passed 1 out of the required 1 times. Success!
1412test_query_new_column passed 1 out of the required 1 times. Success!
1413test_rename_column passed 1 out of the required 1 times. Success!
1414test_rename_column_atomicity passed 1 out of the required 1 times. Success!
1415test_lwt passed 1 out of the required 1 times. Success!
1416test_interrupt_build_process passed 1 out of the required 1 times. Success!
1417test_mv_with_default_ttl_with_flush passed 1 out of the required 1 times. Success!
1418test_mv_with_default_ttl_without_flush passed 1 out of the required 1 times. Success!
1419test_no_base_column_in_view_pk_complex_timestamp_with_flush passed 1 out of the required 1 times. Success!
1420test_base_column_in_view_pk_complex_timestamp_with_flush passed 1 out of the required 1 times. Success!
1421test_base_column_in_view_pk_complex_timestamp_without_flush passed 1 out of the required 1 times. Success!
1422test_expired_liveness_with_limit_rf1_nodes1 passed 1 out of the required 1 times. Success!
1423test_expired_liveness_with_limit_rf1_nodes3 passed 1 out of the required 1 times. Success!
1424test_expired_liveness_with_limit_rf3 passed 1 out of the required 1 times. Success!
1425test_base_column_in_view_pk_commutative_tombstone_with_flush passed 1 out of the required 1 times. Success!
1426test_base_column_in_view_pk_commutative_tombstone_without_flush passed 1 out of the required 1 times. Success!
1427test_view_tombstone passed 1 out of the required 1 times. Success!
1428test_simple_repair_by_base passed 1 out of the required 1 times. Success!
1429test_simple_repair_by_view passed 1 out of the required 1 times. Success!
1430test_base_replica_repair passed 1 out of the required 1 times. Success!
1431test_base_replica_repair_with_contention passed 1 out of the required 1 times. Success!
1432test_complex_mv_select_statements passed 1 out of the required 1 times. Success!
1433test_base_view_consistency_on_failure_after_mv_apply passed 1 out of the required 1 times. Success!
1434test_base_view_consistency_on_failure_before_mv_apply passed 1 out of the required 1 times. Success!
1435test_multi_partition_consistent_reads_after_write passed 1 out of the required 1 times. Success!
1436test_mutations_dontblock passed 1 out of the required 1 times. Success!
1437test_putget_2dc_rf1 passed 1 out of the required 1 times. Success!
1438test_putget_2dc_rf2 passed 1 out of the required 1 times. Success!
1439test_connect_to_ssl passed 1 out of the required 1 times. Success!
1440test_connect_to_ssl_optional passed 1 out of the required 1 times. Success!
1441test_use_custom_port passed 1 out of the required 1 times. Success!
1442test_use_custom_ssl_port passed 1 out of the required 1 times. Success!
1443test_decommission_after_drain_is_invalid passed 1 out of the required 1 times. Success!
1444test_correct_dc_rack_in_nodetool_info passed 1 out of the required 1 times. Success!
1445test_nodetool_timeout_commands passed 1 out of the required 1 times. Success!
1446test_cleanup_when_no_replica_with_index passed 1 out of the required 1 times. Success!
1447test_cleanup_when_no_replica_without_index passed 1 out of the required 1 times. Success!
1448test_meaningless_notice_in_status passed 1 out of the required 1 times. Success!
1449test_reloadlocalschema passed 1 out of the required 1 times. Success!
1450test_refresh_size_estimates_clears_invalid_entries passed 1 out of the required 1 times. Success!
1451test_sstablelevelreset passed 1 out of the required 1 times. Success!
1452test_sstableofflinerelevel passed 1 out of the required 1 times. Success!
1453test_sstableverify failed and was not selected for rerun.
1454	<class 'AssertionError'>
1455	assert None
1456 +  where None = <function search at 0x106e461e0>(('Corrupted: ' + '/var/folders/g3/9njfx8nd2f91w7kcbzxnhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db'), "Subprocess sstableverify on keyspace1 : standard1 with options: ['-v'] exited with non-zero status; exit status: 1; \...nhbnxwd6p5v/T/dtest-ofc8yw3a/test/node1/data1/keyspace1/standard1-ccc09c00284911ea90cd2f4e8a5a8d5e/md-14-big-Data.db\n")
1457 +    where <function search at 0x106e461e0> = re.search
1458	[<TracebackEntry /Users/asp/development/cassandra-dtest/offline_tools_test.py:296>]
1459test_sstableexpiredblockers passed 1 out of the required 1 times. Success!
1460test_sstableupgrade passed 1 out of the required 1 times. Success!
1461test_sstabledump passed 1 out of the required 1 times. Success!
1462test_with_no_results passed 1 out of the required 1 times. Success!
1463test_with_less_results_than_page_size passed 1 out of the required 1 times. Success!
1464test_with_more_results_than_page_size passed 1 out of the required 1 times. Success!
1465test_with_equal_results_to_page_size passed 1 out of the required 1 times. Success!
1466test_undefined_page_size_default passed 1 out of the required 1 times. Success!
1467test_with_order_by passed 1 out of the required 1 times. Success!
1468test_with_order_by_reversed passed 1 out of the required 1 times. Success!
1469test_with_limit passed 1 out of the required 1 times. Success!
1470test_with_allow_filtering passed 1 out of the required 1 times. Success!
1471test_paging_a_single_wide_row passed 1 out of the required 1 times. Success!
1472test_paging_across_multi_wide_rows passed 1 out of the required 1 times. Success!
1473test_paging_using_secondary_indexes passed 1 out of the required 1 times. Success!
1474test_paging_with_in_orderby_and_two_partition_keys passed 1 out of the required 1 times. Success!
1475test_group_by_paging passed 1 out of the required 1 times. Success!
1476test_group_by_with_range_name_query_paging passed 1 out of the required 1 times. Success!
1477test_group_by_with_static_columns_paging passed 1 out of the required 1 times. Success!
1478test_static_columns_paging passed 1 out of the required 1 times. Success!
1479test_paging_using_secondary_indexes_with_static_cols passed 1 out of the required 1 times. Success!
1480test_static_columns_with_empty_non_static_columns_paging passed 1 out of the required 1 times. Success!
1481test_select_in_clause_with_duplicate_keys passed 1 out of the required 1 times. Success!
1482test_paging_with_filtering passed 1 out of the required 1 times. Success!
1483test_paging_with_filtering_on_counter_columns passed 1 out of the required 1 times. Success!
1484test_paging_with_filtering_on_counter_columns_compact passed 1 out of the required 1 times. Success!
1485test_paging_with_filtering_on_clustering_columns passed 1 out of the required 1 times. Success!
1486test_paging_with_filtering_on_clustering_columns_compact passed 1 out of the required 1 times. Success!
1487test_paging_with_filtering_on_clustering_columns_with_contains passed 1 out of the required 1 times. Success!
1488test_paging_with_filtering_on_static_columns passed 1 out of the required 1 times. Success!
1489test_paging_with_filtering_on_partition_key passed 1 out of the required 1 times. Success!
1490test_paging_with_filtering_on_partition_key_with_limit passed 1 out of the required 1 times. Success!
1491test_paging_with_filtering_on_partition_key_on_counter_columns passed 1 out of the required 1 times. Success!
1492test_paging_with_filtering_on_partition_key_on_counter_columns_compact passed 1 out of the required 1 times. Success!
1493test_paging_with_filtering_on_partition_key_on_clustering_columns passed 1 out of the required 1 times. Success!
1494test_paging_with_filtering_on_partition_key_on_clustering_columns_compact passed 1 out of the required 1 times. Success!
1495test_paging_with_filtering_on_partition_key_on_clustering_columns_with_contains passed 1 out of the required 1 times. Success!
1496test_paging_with_filtering_on_partition_key_on_static_columns passed 1 out of the required 1 times. Success!
1497test_paging_on_compact_table_with_tombstone_on_first_column passed 1 out of the required 1 times. Success!
1498test_paging_with_no_clustering_columns passed 1 out of the required 1 times. Success!
1499test_paging_with_no_clustering_columns_compact passed 1 out of the required 1 times. Success!
1500test_per_partition_limit_paging passed 1 out of the required 1 times. Success!
1501test_paging_for_range_name_queries passed 1 out of the required 1 times. Success!
1502test_paging_for_range_name_queries_compact passed 1 out of the required 1 times. Success!
1503test_paging_with_empty_row_and_empty_static_columns passed 1 out of the required 1 times. Success!
1504test_data_change_impacting_earlier_page passed 1 out of the required 1 times. Success!
1505test_data_change_impacting_later_page passed 1 out of the required 1 times. Success!
1506test_row_TTL_expiry_during_paging passed 1 out of the required 1 times. Success!
1507test_cell_TTL_expiry_during_paging passed 1 out of the required 1 times. Success!
1508test_node_unavailabe_during_paging passed 1 out of the required 1 times. Success!
1509test_query_isolation passed 1 out of the required 1 times. Success!
1510test_single_partition_deletions passed 1 out of the required 1 times. Success!
1511test_multiple_partition_deletions passed 1 out of the required 1 times. Success!
1512test_single_row_deletions passed 1 out of the required 1 times. Success!
1513test_single_cell_deletions passed 1 out of the required 1 times. Success!
1514test_multiple_cell_deletions passed 1 out of the required 1 times. Success!
1515test_ttl_deletions passed 1 out of the required 1 times. Success!
1516test_failure_threshold_deletions passed 1 out of the required 1 times. Success!
1517test_deletion_with_distinct_paging passed 1 out of the required 1 times. Success!
1518test_replica_availability passed 1 out of the required 1 times. Success!
1519test_cluster_availability passed 1 out of the required 1 times. Success!
1520test_contention_many_threads passed 1 out of the required 1 times. Success!
1521test_dropped_index passed 1 out of the required 1 times. Success!
1522test_move_single_node passed 1 out of the required 1 times. Success!
1523test_move_single_node_localhost passed 1 out of the required 1 times. Success!
1524test_restart_node passed 1 out of the required 1 times. Success!
1525test_restart_node_localhost passed 1 out of the required 1 times. Success!
1526test_add_and_remove_node passed 1 out of the required 1 times. Success!
1527test_schema_changes passed 1 out of the required 1 times. Success!
1528test_tombstone_failure_threshold_message passed 1 out of the required 1 times. Success!
1529test_putget passed 1 out of the required 1 times. Success!
1530test_putget_snappy passed 1 out of the required 1 times. Success!
1531test_putget_deflate passed 1 out of the required 1 times. Success!
1532test_non_local_read passed 1 out of the required 1 times. Success!
1533test_rangeputget passed 1 out of the required 1 times. Success!
1534test_wide_row passed 1 out of the required 1 times. Success!
1535test_wide_slice passed 1 out of the required 1 times. Success!
1536test_ghosts passed 1 out of the required 1 times. Success!
1537test_tombstone_failure_v3 passed 1 out of the required 1 times. Success!
1538test_tombstone_failure_v4 passed 1 out of the required 1 times. Success!
1539test_tombstone_failure_v5 passed 1 out of the required 1 times. Success!
1540test_alter_rf_and_run_read_repair passed 1 out of the required 1 times. Success!
1541test_read_repair_chance passed 1 out of the required 1 times. Success!
1542test_range_slice_query_with_tombstones passed 1 out of the required 1 times. Success!
1543test_gcable_tombstone_resurrection_on_range_slice_query passed 1 out of the required 1 times. Success!
1544test_simple_rebuild passed 1 out of the required 1 times. Success!
1545test_resumable_rebuild passed 1 out of the required 1 times. Success!
1546test_rebuild_ranges passed 1 out of the required 1 times. Success!
1547test_disallow_rebuild_nonlocal_range passed 1 out of the required 1 times. Success!
1548test_disallow_rebuild_from_nonreplica passed 1 out of the required 1 times. Success!
1549test_rebuild_with_specific_sources passed 1 out of the required 1 times. Success!
1550test_refresh_deadlock_startup passed 1 out of the required 1 times. Success!
1551test_fail_without_replace passed 1 out of the required 1 times. Success!
1552test_unsafe_replace passed 1 out of the required 1 times. Success!
1553test_insert_data_during_replace_same_address passed 1 out of the required 1 times. Success!
1554test_insert_data_during_replace_different_address passed 1 out of the required 1 times. Success!
1555test_replace_with_insufficient_replicas passed 1 out of the required 1 times. Success!
1556test_simple passed 1 out of the required 1 times. Success!
1557test_rf_collapse_gossiping_property_file_snitch passed 1 out of the required 1 times. Success!
1558test_rf_expand_gossiping_property_file_snitch passed 1 out of the required 1 times. Success!
1559test_rf_collapse_property_file_snitch passed 1 out of the required 1 times. Success!
1560test_rf_expand_property_file_snitch passed 1 out of the required 1 times. Success!
1561test_cannot_restart_with_different_rack passed 1 out of the required 1 times. Success!
1562test_failed_snitch_update_gossiping_property_file_snitch passed 1 out of the required 1 times. Success!
1563test_failed_snitch_update_property_file_snitch passed 1 out of the required 1 times. Success!
1564test_switch_data_center_startup_fails passed 1 out of the required 1 times. Success!
1565test_creating_and_dropping_keyspace passed 1 out of the required 1 times. Success!
1566test_creating_and_dropping_table passed 1 out of the required 1 times. Success!
1567test_creating_and_dropping_table_with_2ary_indexes passed 1 out of the required 1 times. Success!
1568test_creating_and_dropping_user_types passed 1 out of the required 1 times. Success!
1569test_creating_and_dropping_udf passed 1 out of the required 1 times. Success!
1570test_creating_and_dropping_uda passed 1 out of the required 1 times. Success!
1571test_basic_table_datatype passed 1 out of the required 1 times. Success!
1572test_collection_table_datatype passed 1 out of the required 1 times. Success!
1573test_clustering_order passed 1 out of the required 1 times. Success!
1574test_compact_storage passed 1 out of the required 1 times. Success!
1575test_compact_storage_composite passed 1 out of the required 1 times. Success!
1576test_nondefault_table_settings passed 1 out of the required 1 times. Success!
1577test_indexes passed 1 out of the required 1 times. Success!
1578test_durable_writes passed 1 out of the required 1 times. Success!
1579test_static_column passed 1 out of the required 1 times. Success!
1580test_udt_table passed 1 out of the required 1 times. Success!
1581test_udf passed 1 out of the required 1 times. Success!
1582test_uda passed 1 out of the required 1 times. Success!
1583test_table_alteration passed 1 out of the required 1 times. Success!
1584test_drop_column_compact passed 1 out of the required 1 times. Success!
1585test_drop_column_compaction passed 1 out of the required 1 times. Success!
1586test_drop_column_queries passed 1 out of the required 1 times. Success!
1587test_drop_column_and_restart passed 1 out of the required 1 times. Success!
1588test_drop_static_column_and_restart passed 1 out of the required 1 times. Success!
1589test_scrub_static_table passed 1 out of the required 1 times. Success!
1590test_standalone_scrub passed 1 out of the required 1 times. Success!
1591test_scrub_collections_table passed 1 out of the required 1 times. Success!
1592test_nodetool_scrub passed 1 out of the required 1 times. Success!
1593test_standalone_scrub passed 1 out of the required 1 times. Success!
1594test_standalone_scrub_essential_files_only passed 1 out of the required 1 times. Success!
1595test_scrub_with_UDT passed 1 out of the required 1 times. Success!
1596test_data_created_before_index_not_returned_in_where_query passed 1 out of the required 1 times. Success!
1597test_low_cardinality_indexes passed 1 out of the required 1 times. Success!
1598test_6924_dropping_ks passed 1 out of the required 1 times. Success!
1599test_6924_dropping_cf passed 1 out of the required 1 times. Success!
1600test_8280_validate_indexed_values passed 1 out of the required 1 times. Success!
1601test_8280_validate_indexed_values_compact passed 1 out of the required 1 times. Success!
1602test_manual_rebuild_index passed 1 out of the required 1 times. Success!
1603test_multi_index_filtering_query passed 1 out of the required 1 times. Success!
1604test_only_coordinator_chooses_index_for_query passed 1 out of the required 1 times. Success!
1605test_tuple_indexes passed 1 out of the required 1 times. Success!
1606test_list_indexes passed 1 out of the required 1 times. Success!
1607test_set_indexes passed 1 out of the required 1 times. Success!
1608test_multiple_indexes_on_single_map_column passed 1 out of the required 1 times. Success!
1609test_map_indexes passed 1 out of the required 1 times. Success!
1610test_bootstrap passed 1 out of the required 1 times. Success!
1611test_resume passed 1 out of the required 1 times. Success!
1612test_manual_join passed 1 out of the required 1 times. Success!
1613test_write_survey passed 1 out of the required 1 times. Success!
1614test_startup_no_live_seeds passed 1 out of the required 1 times. Success!
1615test_startup_non_seed_with_peers passed 1 out of the required 1 times. Success!
1616test_startup_after_ring_delay passed 1 out of the required 1 times. Success!
1617test_basic_snapshot_and_restore passed 1 out of the required 1 times. Success!
1618test_snapshot_and_restore_drop_table_remove_dropped_column passed 1 out of the required 1 times. Success!
1619test_snapshot_and_restore_dropping_a_column passed 1 out of the required 1 times. Success!
1620test_archive_commitlog passed 1 out of the required 1 times. Success!
1621test_archive_commitlog_with_active_commitlog passed 1 out of the required 1 times. Success!
1622test_dont_archive_commitlog passed 1 out of the required 1 times. Success!
1623test_archive_commitlog_point_in_time passed 1 out of the required 1 times. Success!
1624test_archive_commitlog_point_in_time_with_active_commitlog passed 1 out of the required 1 times. Success!
1625test_archive_commitlog_point_in_time_with_active_commitlog_ln passed 1 out of the required 1 times. Success!
1626test_archive_and_restore_commitlog_repeatedly passed 1 out of the required 1 times. Success!
1627test_prefer_local_reconnect_on_listen_address passed 1 out of the required 1 times. Success!
1628test_ssl_enabled passed 1 out of the required 1 times. Success!
1629test_ssl_correct_hostname_with_validation passed 1 out of the required 1 times. Success!
1630test_ssl_wrong_hostname_no_validation passed 1 out of the required 1 times. Success!
1631test_ssl_wrong_hostname_with_validation passed 1 out of the required 1 times. Success!
1632test_ssl_client_auth_required_fail passed 1 out of the required 1 times. Success!
1633test_ssl_client_auth_required_succeed passed 1 out of the required 1 times. Success!
1634test_ca_mismatch passed 1 out of the required 1 times. Success!
1635test_sstableloader_uppercase_keyspace_name passed 1 out of the required 1 times. Success!
1636test_incompressible_data_in_compressed_table passed 1 out of the required 1 times. Success!
1637test_remove_index_file passed 1 out of the required 1 times. Success!
1638test_sstableloader_with_mv passed 1 out of the required 1 times. Success!
1639test_split passed 1 out of the required 1 times. Success!
1640test_single_file_split passed 1 out of the required 1 times. Success!
1641test_compaction passed 1 out of the required 1 times. Success!
1642test_abortedcompaction passed 1 out of the required 1 times. Success!
1643test_uniform_ratio passed 1 out of the required 1 times. Success!
1644test_fixed_ratio passed 1 out of the required 1 times. Success!
1645test_quick_write passed 1 out of the required 1 times. Success!
1646test_sc_with_row_cache passed 1 out of the required 1 times. Success!
1647test_functional passed 1 out of the required 1 times. Success!
1648test_local_system_keyspaces passed 1 out of the required 1 times. Success!
1649test_replicated_system_keyspaces passed 1 out of the required 1 times. Success!
1650test_closing_connections passed 1 out of the required 1 times. Success!
1651test_insert passed 1 out of the required 1 times. Success!
1652test_empty_slice passed 1 out of the required 1 times. Success!
1653test_cas passed 1 out of the required 1 times. Success!
1654test_missing_super passed 1 out of the required 1 times. Success!
1655test_count passed 1 out of the required 1 times. Success!
1656test_count_paging passed 1 out of the required 1 times. Success!
1657test_count_around_page_size passed 1 out of the required 1 times. Success!
1658test_super_insert passed 1 out of the required 1 times. Success!
1659test_super_get passed 1 out of the required 1 times. Success!
1660test_super_subcolumn_limit passed 1 out of the required 1 times. Success!
1661test_long_order passed 1 out of the required 1 times. Success!
1662test_integer_order passed 1 out of the required 1 times. Success!
1663test_time_uuid passed 1 out of the required 1 times. Success!
1664test_long_remove passed 1 out of the required 1 times. Success!
1665test_integer_remove passed 1 out of the required 1 times. Success!
1666test_batch_insert passed 1 out of the required 1 times. Success!
1667test_batch_mutate_standard_columns passed 1 out of the required 1 times. Success!
1668test_batch_mutate_remove_standard_columns passed 1 out of the required 1 times. Success!
1669test_batch_mutate_remove_standard_row passed 1 out of the required 1 times. Success!
1670test_batch_mutate_remove_super_columns_with_standard_under passed 1 out of the required 1 times. Success!
1671test_batch_mutate_remove_super_columns_with_none_given_underneath passed 1 out of the required 1 times. Success!
1672test_batch_mutate_remove_super_columns_entire_row passed 1 out of the required 1 times. Success!
1673test_batch_mutate_remove_slice_standard passed 1 out of the required 1 times. Success!
1674test_batch_mutate_remove_slice_of_entire_supercolumns passed 1 out of the required 1 times. Success!
1675test_batch_mutate_insertions_and_deletions passed 1 out of the required 1 times. Success!
1676test_bad_system_calls passed 1 out of the required 1 times. Success!
1677test_bad_batch_calls passed 1 out of the required 1 times. Success!
1678test_column_name_lengths passed 1 out of the required 1 times. Success!
1679test_bad_calls passed 1 out of the required 1 times. Success!
1680test_batch_insert_super passed 1 out of the required 1 times. Success!
1681test_cf_remove_column passed 1 out of the required 1 times. Success!
1682test_cf_remove passed 1 out of the required 1 times. Success!
1683test_super_cf_remove_and_range_slice passed 1 out of the required 1 times. Success!
1684test_super_cf_remove_column passed 1 out of the required 1 times. Success!
1685test_super_cf_remove_supercolumn passed 1 out of the required 1 times. Success!
1686test_super_cf_resurrect_subcolumn passed 1 out of the required 1 times. Success!
1687test_empty_range passed 1 out of the required 1 times. Success!
1688test_super_cql_read_compatibility passed 1 out of the required 1 times. Success!
1689test_super_cql_write_compatibility passed 1 out of the required 1 times. Success!
1690test_range_with_remove passed 1 out of the required 1 times. Success!
1691test_range_with_remove_cf passed 1 out of the required 1 times. Success!
1692test_range_collation passed 1 out of the required 1 times. Success!
1693test_range_partial passed 1 out of the required 1 times. Success!
1694test_get_slice_range passed 1 out of the required 1 times. Success!
1695test_get_slice_super_range passed 1 out of the required 1 times. Success!
1696test_get_range_slices_tokens passed 1 out of the required 1 times. Success!
1697test_get_range_slice_super passed 1 out of the required 1 times. Success!
1698test_get_range_slice passed 1 out of the required 1 times. Success!
1699test_wrapped_range_slices passed 1 out of the required 1 times. Success!
1700test_get_slice_by_names passed 1 out of the required 1 times. Success!
1701test_multiget_slice_with_compact_table passed 1 out of the required 1 times. Success!
1702test_multiget_slice passed 1 out of the required 1 times. Success!
1703test_multi_count passed 1 out of the required 1 times. Success!
1704test_batch_mutate_super_deletion passed 1 out of the required 1 times. Success!
1705test_super_reinsert passed 1 out of the required 1 times. Success!
1706test_describe_keyspace passed 1 out of the required 1 times. Success!
1707test_describe passed 1 out of the required 1 times. Success!
1708test_describe_ring passed 1 out of the required 1 times. Success!
1709test_describe_token_map passed 1 out of the required 1 times. Success!
1710test_describe_partitioner passed 1 out of the required 1 times. Success!
1711test_describe_snitch passed 1 out of the required 1 times. Success!
1712test_invalid_ks_names passed 1 out of the required 1 times. Success!
1713test_invalid_strategy_class passed 1 out of the required 1 times. Success!
1714test_invalid_cf_names passed 1 out of the required 1 times. Success!
1715test_system_cf_recreate passed 1 out of the required 1 times. Success!
1716test_system_keyspace_operations passed 1 out of the required 1 times. Success!
1717test_create_then_drop_ks passed 1 out of the required 1 times. Success!
1718test_column_validators passed 1 out of the required 1 times. Success!
1719test_system_column_family_operations passed 1 out of the required 1 times. Success!
1720test_dynamic_indexes_creation_deletion passed 1 out of the required 1 times. Success!
1721test_dynamic_indexes_with_system_update_cf passed 1 out of the required 1 times. Success!
1722test_system_super_column_family_operations passed 1 out of the required 1 times. Success!
1723test_insert_ttl passed 1 out of the required 1 times. Success!
1724test_insert_max_ttl passed 1 out of the required 1 times. Success!
1725test_insert_max_default_ttl passed 1 out of the required 1 times. Success!
1726test_simple_expiration passed 1 out of the required 1 times. Success!
1727test_expiration_with_default_ttl passed 1 out of the required 1 times. Success!
1728test_expiration_with_default_ttl_and_zero_ttl passed 1 out of the required 1 times. Success!
1729test_simple_expiration_batch_mutate passed 1 out of the required 1 times. Success!
1730test_update_expiring passed 1 out of the required 1 times. Success!
1731test_remove_expiring passed 1 out of the required 1 times. Success!
1732test_describe_ring_on_invalid_keyspace passed 1 out of the required 1 times. Success!
1733test_incr_decr_standard_add passed 1 out of the required 1 times. Success!
1734test_incr_decr_super_add passed 1 out of the required 1 times. Success!
1735test_incr_standard_remove passed 1 out of the required 1 times. Success!
1736test_incr_super_remove passed 1 out of the required 1 times. Success!
1737test_incr_decr_standard_remove passed 1 out of the required 1 times. Success!
1738test_incr_decr_super_remove passed 1 out of the required 1 times. Success!
1739test_incr_decr_standard_batch_add passed 1 out of the required 1 times. Success!
1740test_incr_decr_standard_batch_remove passed 1 out of the required 1 times. Success!
1741test_range_deletion passed 1 out of the required 1 times. Success!
1742test_range_deletion_eoc_0 passed 1 out of the required 1 times. Success!
1743test_incr_decr_standard_slice passed 1 out of the required 1 times. Success!
1744test_incr_decr_standard_multiget_slice passed 1 out of the required 1 times. Success!
1745test_counter_get_slice_range passed 1 out of the required 1 times. Success!
1746test_counter_get_slice_super_range passed 1 out of the required 1 times. Success!
1747test_index_scan passed 1 out of the required 1 times. Success!
1748test_index_scan_uuid_names passed 1 out of the required 1 times. Success!
1749test_index_scan_expiring passed 1 out of the required 1 times. Success!
1750test_index_scan_indexed_column_outside_slice_predicate passed 1 out of the required 1 times. Success!
1751test_column_not_found_quorum passed 1 out of the required 1 times. Success!
1752test_get_range_slice_after_deletion passed 1 out of the required 1 times. Success!
1753test_multi_slice passed 1 out of the required 1 times. Success!
1754test_truncate passed 1 out of the required 1 times. Success!
1755test_cql_range_tombstone_and_static passed 1 out of the required 1 times. Success!
1756test_compact_storage_get passed 1 out of the required 1 times. Success!
1757test_range_tombstone_eoc_0 passed 1 out of the required 1 times. Success!
1758test_do_not_join_ring passed 1 out of the required 1 times. Success!
1759test_size_estimates_multidc passed 1 out of the required 1 times. Success!
1760test_simple_decommission passed 1 out of the required 1 times. Success!
1761test_resumable_decommission passed 1 out of the required 1 times. Success!
1762test_movement passed 1 out of the required 1 times. Success!
1763test_decommission passed 1 out of the required 1 times. Success!
1764test_move_single_node passed 1 out of the required 1 times. Success!
1765test_decommissioned_node_cant_rejoin passed 1 out of the required 1 times. Success!
1766test_crash_during_decommission passed 1 out of the required 1 times. Success!
1767test_default_ttl passed 1 out of the required 1 times. Success!
1768test_insert_ttl_has_priority_on_defaut_ttl passed 1 out of the required 1 times. Success!
1769test_insert_ttl_works_without_default_ttl passed 1 out of the required 1 times. Success!
1770test_default_ttl_can_be_removed passed 1 out of the required 1 times. Success!
1771test_removing_default_ttl_does_not_affect_existing_rows passed 1 out of the required 1 times. Success!
1772test_update_single_column_ttl passed 1 out of the required 1 times. Success!
1773test_update_multiple_columns_ttl passed 1 out of the required 1 times. Success!
1774test_update_column_ttl_with_default_ttl passed 1 out of the required 1 times. Success!
1775test_remove_column_ttl passed 1 out of the required 1 times. Success!
1776test_set_ttl_to_zero_to_default_ttl passed 1 out of the required 1 times. Success!
1777test_collection_list_ttl passed 1 out of the required 1 times. Success!
1778test_collection_set_ttl passed 1 out of the required 1 times. Success!
1779test_collection_map_ttl passed 1 out of the required 1 times. Success!
1780test_delete_with_ttl_expired passed 1 out of the required 1 times. Success!
1781test_expiration_overflow_policy_cap passed 1 out of the required 1 times. Success!
1782test_expiration_overflow_policy_cap_default_ttl passed 1 out of the required 1 times. Success!
1783test_expiration_overflow_policy_capnowarn passed 1 out of the required 1 times. Success!
1784test_expiration_overflow_policy_capnowarn_default_ttl passed 1 out of the required 1 times. Success!
1785test_expiration_overflow_policy_reject passed 1 out of the required 1 times. Success!
1786test_expiration_overflow_policy_reject_default_ttl passed 1 out of the required 1 times. Success!
1787test_ttl_is_replicated passed 1 out of the required 1 times. Success!
1788test_ttl_is_respected_on_delayed_replication passed 1 out of the required 1 times. Success!
1789test_ttl_is_respected_on_repair passed 1 out of the required 1 times. Success!
1790test_recover_negative_expiration_date_sstables_with_scrub passed 1 out of the required 1 times. Success!
1791test_udt passed 1 out of the required 1 times. Success!
1792test_migration passed 1 out of the required 1 times. Success!
1793test_udf_overload passed 1 out of the required 1 times. Success!
1794test_udf_scripting passed 1 out of the required 1 times. Success!
1795test_default_aggregate passed 1 out of the required 1 times. Success!
1796test_aggregate_udf passed 1 out of the required 1 times. Success!
1797test_udf_with_udt passed 1 out of the required 1 times. Success!
1798test_udf_with_udt_keyspace_isolation passed 1 out of the required 1 times. Success!
1799test_aggregate_with_udt_keyspace_isolation passed 1 out of the required 1 times. Success!
1800test_type_dropping passed 1 out of the required 1 times. Success!
1801test_nested_type_dropping passed 1 out of the required 1 times. Success!
1802test_type_enforcement passed 1 out of the required 1 times. Success!
1803test_nested_user_types passed 1 out of the required 1 times. Success!
1804test_type_as_part_of_pkey passed 1 out of the required 1 times. Success!
1805test_type_secondary_indexing passed 1 out of the required 1 times. Success!
1806test_type_keyspace_permission_isolation passed 1 out of the required 1 times. Success!
1807test_nulls_in_user_types passed 1 out of the required 1 times. Success!
1808test_no_counters_in_user_types passed 1 out of the required 1 times. Success!
1809test_type_as_clustering_col passed 1 out of the required 1 times. Success!
1810test_udt_subfield passed 1 out of the required 1 times. Success!
1811test_user_type_isolation passed 1 out of the required 1 times. Success!
1812test_wide_rows passed 1 out of the required 1 times. Success!
1813test_column_index_stress passed 1 out of the required 1 times. Success!
1814test_mutation_v3 passed 1 out of the required 1 times. Success!
1815test_mutation_v4 passed 1 out of the required 1 times. Success!
1816test_mutation_any passed 1 out of the required 1 times. Success!
1817test_mutation_one passed 1 out of the required 1 times. Success!
1818test_mutation_quorum passed 1 out of the required 1 times. Success!
1819test_batch passed 1 out of the required 1 times. Success!
1820test_counter passed 1 out of the required 1 times. Success!
1821test_paxos passed 1 out of the required 1 times. Success!
1822test_paxos_any passed 1 out of the required 1 times. Success!
1823test_thrift passed 1 out of the required 1 times. Success!
1824test_simple_insert passed 1 out of the required 1 times. Success!
1825test_lwt passed 1 out of the required 1 times. Success!
1826test_past_and_future_dates passed 1 out of the required 1 times. Success!
1827test_sub_second_precision passed 1 out of the required 1 times. Success!
1828test_eat_glass passed 1 out of the required 1 times. Success!
1829test_source_glass passed 1 out of the required 1 times. Success!
1830test_unicode_syntax_error passed 1 out of the required 1 times. Success!
1831test_unicode_invalid_request_error passed 1 out of the required 1 times. Success!
1832test_with_empty_values passed 1 out of the required 1 times. Success!
1833test_tracing_from_system_traces passed 1 out of the required 1 times. Success!
1834test_select_element_inside_udt passed 1 out of the required 1 times. Success!
1835test_list_queries passed 1 out of the required 1 times. Success!
1836test_describe passed 1 out of the required 1 times. Success!
1837test_describe_describes_non_default_compaction_parameters passed 1 out of the required 1 times. Success!
1838test_describe_on_non_reserved_keywords passed 1 out of the required 1 times. Success!
1839test_describe_mv passed 1 out of the required 1 times. Success!
1840test_copy_to passed 1 out of the required 1 times. Success!
1841test_float_formatting passed 1 out of the required 1 times. Success!
1842test_int_values passed 1 out of the required 1 times. Success!
1843test_datetime_values passed 1 out of the required 1 times. Success!
1844test_tracing passed 1 out of the required 1 times. Success!
1845test_client_warnings passed 1 out of the required 1 times. Success!
1846test_connect_timeout passed 1 out of the required 1 times. Success!
1847test_protocol_negotiation passed 1 out of the required 1 times. Success!
1848test_protocol_version_restriction passed 1 out of the required 1 times. Success!
1849test_update_schema_with_down_node passed 1 out of the required 1 times. Success!
1850test_describe_round_trip passed 1 out of the required 1 times. Success!
1851test_materialized_view passed 1 out of the required 1 times. Success!
1852test_clear passed 1 out of the required 1 times. Success!
1853test_cls passed 1 out of the required 1 times. Success!
1854test_batch passed 1 out of the required 1 times. Success!
1855test_uuid passed 1 out of the required 1 times. Success!
1856test_commented_lines passed 1 out of the required 1 times. Success!
1857test_colons_in_string_literals passed 1 out of the required 1 times. Success!
1858test_select passed 1 out of the required 1 times. Success!
1859test_insert passed 1 out of the required 1 times. Success!
1860test_update passed 1 out of the required 1 times. Success!
1861test_delete passed 1 out of the required 1 times. Success!
1862test_batch passed 1 out of the required 1 times. Success!
1863test_create_keyspace passed 1 out of the required 1 times. Success!
1864test_drop_keyspace passed 1 out of the required 1 times. Success!
1865test_create_table passed 1 out of the required 1 times. Success!
1866test_drop_table passed 1 out of the required 1 times. Success!
1867test_truncate passed 1 out of the required 1 times. Success!
1868test_use_keyspace passed 1 out of the required 1 times. Success!
1869test_drop_index passed 1 out of the required 1 times. Success!
1870test_create_index passed 1 out of the required 1 times. Success!
1871test_login_keeps_keyspace passed 1 out of the required 1 times. Success!
1872test_login_rejects_bad_pass passed 1 out of the required 1 times. Success!
1873test_login_authenticates_correct_user passed 1 out of the required 1 times. Success!
1874test_login_allows_bad_pass_and_continued_use passed 1 out of the required 1 times. Success!
1875test_list_roles_after_login passed 1 out of the required 1 times. Success!
1876test_reading_collections_with_empty_values passed 1 out of the required 1 times. Success!
1877test_writing_use_header passed 1 out of the required 1 times. Success!
1878test_reading_counter passed 1 out of the required 1 times. Success!
1879test_reading_counter_without_batching passed 1 out of the required 1 times. Success!
1880test_reading_use_header passed 1 out of the required 1 times. Success!
1881test_reading_with_ttl passed 1 out of the required 1 times. Success!
1882test_reading_with_skip_and_max_rows passed 1 out of the required 1 times. Success!
1883test_reading_with_skip_cols passed 1 out of the required 1 times. Success!
1884test_reading_counters_with_skip_cols passed 1 out of the required 1 times. Success!
1885test_writing_with_token_boundaries passed 1 out of the required 1 times. Success!
1886test_reading_max_parse_errors passed 1 out of the required 1 times. Success!
1887test_reading_max_insert_errors passed 1 out of the required 1 times. Success!
1888test_reading_with_multiple_files passed 1 out of the required 1 times. Success!
1889test_writing_with_max_output_size passed 1 out of the required 1 times. Success!
1890test_explicit_column_order_writing passed 1 out of the required 1 times. Success!
1891test_quoted_column_names_writing passed 1 out of the required 1 times. Success!
1892test_read_wrong_column_names passed 1 out of the required 1 times. Success!
1893test_boolstyle_round_trip passed 1 out of the required 1 times. Success!
1894test_round_trip_with_sub_second_precision passed 1 out of the required 1 times. Success!
1895test_round_trip_with_different_number_precision passed 1 out of the required 1 times. Success!
1896test_round_trip_with_num_processes passed 1 out of the required 1 times. Success!
1897test_round_trip_with_rate_file passed 1 out of the required 1 times. Success!
1898test_copy_options_from_config_file passed 1 out of the required 1 times. Success!
1899test_wrong_number_of_columns passed 1 out of the required 1 times. Success!
1900test_round_trip_murmur3 passed 1 out of the required 1 times. Success!
1901test_round_trip_random passed 1 out of the required 1 times. Success!
1902test_round_trip_order_preserving passed 1 out of the required 1 times. Success!
1903test_round_trip_byte_ordered passed 1 out of the required 1 times. Success!
1904test_source_copy_round_trip passed 1 out of the required 1 times. Success!
1905test_bulk_round_trip_default passed 1 out of the required 1 times. Success!
1906test_bulk_round_trip_non_prepared_statements passed 1 out of the required 1 times. Success!
1907test_bulk_round_trip_blogposts passed 1 out of the required 1 times. Success!
1908test_bulk_round_trip_blogposts_with_max_connections passed 1 out of the required 1 times. Success!
1909test_bulk_round_trip_with_timeouts passed 1 out of the required 1 times. Success!
1910test_bulk_round_trip_with_low_ingestrate passed 1 out of the required 1 times. Success!
1911test_bulk_round_trip_with_single_core passed 1 out of the required 1 times. Success!
1912test_bulk_round_trip_with_backoff passed 1 out of the required 1 times. Success!
1913test_copy_to_with_more_failures_than_max_attempts passed 1 out of the required 1 times. Success!
1914test_copy_to_with_fewer_failures_than_max_attempts passed 1 out of the required 1 times. Success!
1915test_copy_to_with_child_process_crashing passed 1 out of the required 1 times. Success!
1916test_copy_from_with_more_failures_than_max_attempts passed 1 out of the required 1 times. Success!
1917test_copy_from_with_fewer_failures_than_max_attempts passed 1 out of the required 1 times. Success!
1918test_copy_from_with_child_process_crashing passed 1 out of the required 1 times. Success!
1919test_copy_from_with_unacked_batches passed 1 out of the required 1 times. Success!
1920test_round_trip_with_authentication passed 1 out of the required 1 times. Success!
1921test_reading_pk_timestamps_with_counters passed 1 out of the required 1 times. Success!
1922test_copy_from_with_wrong_order_or_missing_UDT_fields passed 1 out of the required 1 times. Success!
1923test_importing_invalid_data_for_collections passed 1 out of the required 1 times. Success!
1924test_force_repair_async_1 passed 1 out of the required 1 times. Success!
1925test_force_repair_async_2 passed 1 out of the required 1 times. Success!
1926test_force_repair_async_3 passed 1 out of the required 1 times. Success!
1927test_force_repair_range_async_1 passed 1 out of the required 1 times. Success!
1928test_force_repair_range_async_2 passed 1 out of the required 1 times. Success!
1929test_force_repair_range_async_3 passed 1 out of the required 1 times. Success!
1930test_deprecated_repair_error_notification passed 1 out of the required 1 times. Success!
1931test_sstable_marking passed 1 out of the required 1 times. Success!
1932test_multiple_repair passed 1 out of the required 1 times. Success!
1933test_sstable_repairedset passed 1 out of the required 1 times. Success!
1934test_compaction passed 1 out of the required 1 times. Success!
1935test_multiple_full_repairs_lcs passed 1 out of the required 1 times. Success!
1936test_no_anticompaction_after_dclocal_repair passed 1 out of the required 1 times. Success!
1937test_nonexistent_table_repair passed 1 out of the required 1 times. Success!
1938test_no_anticompaction_after_hostspecific_repair passed 1 out of the required 1 times. Success!
1939test_no_anticompaction_after_subrange_repair passed 1 out of the required 1 times. Success!
1940test_no_anticompaction_of_already_repaired passed 1 out of the required 1 times. Success!
1941test_anticompaction_after_normal_repair passed 1 out of the required 1 times. Success!
1942test_simple_sequential_repair passed 1 out of the required 1 times. Success!
1943test_simple_parallel_repair passed 1 out of the required 1 times. Success!
1944test_empty_vs_gcable_sequential_repair passed 1 out of the required 1 times. Success!
1945test_empty_vs_gcable_parallel_repair passed 1 out of the required 1 times. Success!
1946test_range_tombstone_digest_sequential_repair passed 1 out of the required 1 times. Success!
1947test_range_tombstone_digest_parallel_repair passed 1 out of the required 1 times. Success!
1948test_shadowed_cell_digest_sequential_repair passed 1 out of the required 1 times. Success!
1949test_shadowed_cell_digest_parallel_repair passed 1 out of the required 1 times. Success!
1950test_shadowed_range_tombstone_digest_sequential_repair passed 1 out of the required 1 times. Success!
1951test_shadowed_range_tombstone_digest_parallel_repair passed 1 out of the required 1 times. Success!
1952test_simple_repair_order_preserving passed 1 out of the required 1 times. Success!
1953test_local_dc_repair passed 1 out of the required 1 times. Success!
1954test_dc_repair passed 1 out of the required 1 times. Success!
1955test_dc_parallel_repair passed 1 out of the required 1 times. Success!
1956test_token_range_repair passed 1 out of the required 1 times. Success!
1957test_token_range_repair_with_cf passed 1 out of the required 1 times. Success!
1958test_partitioner_range_repair passed 1 out of the required 1 times. Success!
1959test_pull_repair passed 1 out of the required 1 times. Success!
1960test_trace_repair passed 1 out of the required 1 times. Success!
1961test_thread_count_repair passed 1 out of the required 1 times. Success!
1962test_multiple_concurrent_repairs passed 1 out of the required 1 times. Success!
1963test_dead_coordinator passed 1 out of the required 1 times. Success!
1964test_dead_sync_initiator passed 1 out of the required 1 times. Success!
1965test_dead_sync_participant passed 1 out of the required 1 times. Success!
1966test_failure_during_anticompaction passed 1 out of the required 1 times. Success!
1967test_failure_during_validation passed 1 out of the required 1 times. Success!
1968
1969===End Flaky Test Report===
1970=============================== warnings summary ===============================
1971native_transport_ssl_test.py::TestNativeTransportSSL::()::test_use_custom_port
1972  /Users/asp/dtest/src/ccm/ccmlib/node.py:584: UserWarning: Binary interface 127.0.0.1:9042 is not listening after 30 seconds, node may have failed to start.
1973    % (binary_itf[0], binary_itf[1]))
1974
1975-- Docs: http://doc.pytest.org/en/latest/warnings.html
1976= 5 failed, 891 passed, 148 skipped, 4485 deselected, 1 warnings in 31907.99 seconds =