May
18
12 hours ago
open pull request

azat wants to merge ClickHouse/ClickHouse

azat
azat

Add total_max_threads parameter

Changelog category:

  • New Feature

Changelog entry

Add total_max_threads parameter to increase performance in case of high RPS by means of limiting total number of threads for all queries.

Closes #36551

See the attached article for details: article.pdf

azat
azat

Not only disk bound, thread may just poll something once in a ms or so.

pull request

azat merge to ClickHouse/ClickHouse

azat
azat

Add total_max_threads parameter

Changelog category:

  • New Feature

Changelog entry

Add total_max_threads parameter to increase performance in case of high RPS by means of limiting total number of threads for all queries.

Closes #36551

See the attached article for details: article.pdf

open pull request

azat wants to merge ClickHouse/ClickHouse

azat
azat

Add total_max_threads parameter

Changelog category:

  • New Feature

Changelog entry

Add total_max_threads parameter to increase performance in case of high RPS by means of limiting total number of threads for all queries.

Closes #36551

See the attached article for details: article.pdf

azat
azat

To me, functions can be leased as now, since it is already attached to processes, so simply total_max_threads is fine.

pull request

azat merge to ClickHouse/ClickHouse

azat
azat

Add total_max_threads parameter

Changelog category:

  • New Feature

Changelog entry

Add total_max_threads parameter to increase performance in case of high RPS by means of limiting total number of threads for all queries.

Closes #36551

See the attached article for details: article.pdf

push

mergify[bot] push ClickHouse/ClickHouse

mergify[bot]
mergify[bot]

add select implementation for MeiliSearch

mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]

fix code-style & add float type

mergify[bot]
mergify[bot]

rewrite SinkMeiliSearch using JSONRowOutputFormat

mergify[bot]
mergify[bot]

init commit with parsing and BAD realisation

mergify[bot]
mergify[bot]

rm unnessesary data committed by mistake

mergify[bot]
mergify[bot]

remove another unnessesary files

mergify[bot]
mergify[bot]

revert changes made to cube transform

mergify[bot]
mergify[bot]

erase blank line to restore initial state

mergify[bot]
mergify[bot]

feat grouping-sets: initial changes

mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]
mergify[bot]

commit sha: d5f870eac8e459362332f5da56aa3e485d924c19

push time in 7 minutes ago
Activity icon
issue

mergify[bot] issue comment ClickHouse/ClickHouse

mergify[bot]
mergify[bot]

Multiple client connection attempts if hostname resolves to multiple addresses

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Client will try every IP address returned by DNS resolution until successful connection.

closes #6698

mergify[bot]
mergify[bot]

update

✅ Branch has been successfully updated

Activity icon
issue

yakov-olkhovskiy issue comment ClickHouse/ClickHouse

yakov-olkhovskiy
yakov-olkhovskiy

Multiple client connection attempts if hostname resolves to multiple addresses

Changelog category (leave one):

  • Bug Fix (user-visible misbehavior in official stable or prestable release)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Client will try every IP address returned by DNS resolution until successful connection.

closes #6698

pull request

KochetovNicolai merge to ClickHouse/ClickHouse

KochetovNicolai
KochetovNicolai

Speed up test 00157_cache_dictionary

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)
push

KochetovNicolai push ClickHouse/ClickHouse

KochetovNicolai
KochetovNicolai

tests/integration: add logging for NetworkManager

Signed-off-by: Azat Khuzhin [email protected]

KochetovNicolai
KochetovNicolai

tests/integration: use no-resolve and verbose for iptables --list

Signed-off-by: Azat Khuzhin [email protected]

KochetovNicolai
KochetovNicolai

tests/integration: remove superfluous import of PartitionManager

Signed-off-by: Azat Khuzhin [email protected]

KochetovNicolai
KochetovNicolai

tests/integration: fix possible race for iptables user rules inside containers

It is possible for network PartitionManager to work incorrectly, because of how docker setting up forward to DOCKER-USER chain, it first removes forward and then adds it back (see 1 and 2), however this introduce race for a short period of time, and this is enough for TCP to retransmit packets, and breaks network PartitionManager.

Here are some details from logs for 3:

2022-04-27 03:01:00 [ 621 ] DEBUG : Executing query SELECT node FROM distributed_table ORDER BY node on node2 (cluster.py:2879, query_and_get_error)

This query fails, from the server logs:

2022.04.27 03:01:00.213101 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> executeQuery: (from 172.16.5.1:59008) SELECT node FROM distributed_table ORDER BY node
...
2022.04.27 03:01:03.578439 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> Connection (node1:9000): Sent data for 2 scalars, total 2 rows in 0.000284672 sec., 6993 rows/sec., 68.00 B (232.15 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (505.16 KiB/sec.)
2022.04.27 03:01:03.590637 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MergingSortedTransform: Merge sorted 3 blocks, 2 rows in 3.371592744 sec., 0.5931914533744174 rows/sec., 94.61 B/sec
2022.04.27 03:01:03.601256 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Information> executeQuery: Read 2 rows, 28.00 B in 3.387950542 sec., 0 rows/sec., 8.26 B/sec.
2022.04.27 03:01:03.601894 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MemoryTracker: Peak memory usage (for query): 334.38 KiB.

And from docker daemon log:

time="2022-04-27T03:00:59.916693113Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
time="2022-04-27T03:01:00.030654116Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
...
time="2022-04-27T03:01:03.515813984Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -n -L DOCKER-USER]"
time="2022-04-27T03:01:03.531106486Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C DOCKER-USER -j RETURN]"
time="2022-04-27T03:01:03.535442346Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-USER]"
time="2022-04-27T03:01:03.555856911Z" level=debug msg="/usr/sbin/iptables, [--wait -D FORWARD -j DOCKER-USER]"
time="2022-04-27T03:01:03.564905764Z" level=debug msg="/usr/sbin/iptables, [--wait -I FORWARD -j DOCKER-USER]"
...
time="2022-04-27T03:01:03.706374466Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
time="2022-04-27T03:01:03.968077970Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"

I've tried multiple ways of fixing this:

  • Creating separate chain for rules from PartitionManager (DOCKER-USER-CLICKHOUSE) But it is created only once, and docker places new rules on top of the FORWARD chain, so it will not work, since it will not receive any packets

  • Use DOCKER-USER, but replace iptables with a wrapper ([script]), that will ignore recreating of a rule for forward to DOCKER-USER, but this will not work too, since new docker rules will be created on top of FORWARD chain, and so DOCKER-USER will packets.

    [script]:

    if [[ "$" =~ "-D FORWARD -j DOCKER-USER" ]]; then exit 0 fi if [[ "$" =~ "-I FORWARD -j DOCKER-USER" ]]; then if iptables.real iptables -C FORWARD -j DOCKER-USER; then exit 0 fi fi

  • And the only way to avoid flakiness for this case, is to forbid parallel execution for tests with PartitionManager.

Signed-off-by: Azat Khuzhin [email protected]

KochetovNicolai
KochetovNicolai

tests/integration: add prefix match for skipped tests

This way you can specify only file/module, or test name without parameters.

Since, at least one, test that we care about (test_distributed_respect_user_timeouts/test.py::test_reconnect) was not runned sequentially 1.

Signed-off-by: Azat Khuzhin [email protected]

KochetovNicolai
KochetovNicolai

Merge pull request #37138 from azat/integration-tests-iptables

tests/integration: fix possible race for iptables user rules inside containers

commit sha: a19d4c6f1fc49bfb4964c7288ba9373ff908306a

push time in 12 minutes ago
pull request

KochetovNicolai pull request ClickHouse/ClickHouse

KochetovNicolai
KochetovNicolai

tests/integration: fix possible race for iptables user rules inside containers

Changelog category (leave one):

  • Not for changelog (changelog entry is not required)

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

tests/integration: fix possible race for iptables user rules inside containers

TL;DR;

It is possible for network PartitionManager to work incorrectly, because of how docker setting up forward to DOCKER-USER chain, it first removes forward and then adds it back (see 1 and 2), however this introduce race for a short period of time, and this is enough for TCP to retransmit packets, and breaks network PartitionManager.

Here are some details from logs for 3:

2022-04-27 03:01:00 [ 621 ] DEBUG : Executing query SELECT node FROM distributed_table ORDER BY node on node2 (cluster.py:2879, query_and_get_error)

This query fails, from the server logs:

2022.04.27 03:01:00.213101 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> executeQuery: (from 172.16.5.1:59008) SELECT node FROM distributed_table ORDER BY node
...
2022.04.27 03:01:03.578439 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> Connection (node1:9000): Sent data for 2 scalars, total 2 rows in 0.000284672 sec., 6993 rows/sec., 68.00 B (232.15 KiB/sec.), compressed 0.4594594594594595 times to 148.00 B (505.16 KiB/sec.)
2022.04.27 03:01:03.590637 [ 223 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MergingSortedTransform: Merge sorted 3 blocks, 2 rows in 3.371592744 sec., 0.5931914533744174 rows/sec., 94.61 B/sec
2022.04.27 03:01:03.601256 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Information> executeQuery: Read 2 rows, 28.00 B in 3.387950542 sec., 0 rows/sec., 8.26 B/sec.
2022.04.27 03:01:03.601894 [ 10 ] {19b1719f-8c39-4e3e-b782-aa4c933650f2} <Debug> MemoryTracker: Peak memory usage (for query): 334.38 KiB.

And from docker daemon log:

time="2022-04-27T03:00:59.916693113Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
time="2022-04-27T03:01:00.030654116Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-I\",\"DOCKER-USER\",\"1\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
...
time="2022-04-27T03:01:03.515813984Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -n -L DOCKER-USER]"
time="2022-04-27T03:01:03.531106486Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C DOCKER-USER -j RETURN]"
time="2022-04-27T03:01:03.535442346Z" level=debug msg="/usr/sbin/iptables, [--wait -t filter -C FORWARD -j DOCKER-USER]"
time="2022-04-27T03:01:03.555856911Z" level=debug msg="/usr/sbin/iptables, [--wait -D FORWARD -j DOCKER-USER]"
time="2022-04-27T03:01:03.564905764Z" level=debug msg="/usr/sbin/iptables, [--wait -I FORWARD -j DOCKER-USER]"
...
time="2022-04-27T03:01:03.706374466Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.3\",\"-d\",\"172.16.5.2\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"
time="2022-04-27T03:01:03.968077970Z" level=debug msg="form data: {\"AttachStderr\":true,\"AttachStdin\":false,\"AttachStdout\":true,\"Cmd\":[\"iptables\",\"--wait\",\"-D\",\"DOCKER-USER\",\"-p\",\"tcp\",\"-s\",\"172.16.5.2\",\"-d\",\"172.16.5.3\",\"-j\",\"DROP\"],\"Container\":\"b75f3b68cda51386bfbb9cceb67e92c4d217a5a1660bde2470b583cb1f4c7fc4\",\"Privileged\":true,\"Tty\":false,\"User\":\"\"}"

I've tried multiple ways of fixing this:

  • Creating separate chain for rules from PartitionManager (DOCKER-USER-CLICKHOUSE) But it is created only once, and docker places new rules on top of the FORWARD chain, so it will not work, since it will not receive any packets

  • Use DOCKER-USER, but replace iptables with a wrapper ([script]), that will ignore recreating of a rule for forward to DOCKER-USER, but this will not work too, since new docker rules will be created on top of FORWARD chain, and so DOCKER-USER will packets.

    [script]:

    if [[ "$*" =~ "-D FORWARD -j DOCKER-USER" ]]; then
        exit 0
    fi
    if [[ "$*" =~ "-I FORWARD -j DOCKER-USER" ]]; then
        if iptables.real iptables -C FORWARD -j DOCKER-USER; then
            exit 0
        fi
    fi
  • And the only way to avoid flakiness for this case, is to forbid parallel execution for tests with PartitionManager.

Fixes: #36541 (fixes first problem, everything else, had been already fixed) Refs: https://github.com/moby/moby/pull/43585

More CI:

Previous