uablrek

uablrek

SW developer at Ericsson Software Technology. IP networking and load balancing.

Member Since 4 years ago

Ericsson, Sweden

Experience Points
29
follower
Lessons Completed
1
follow
Lessons Completed
1
stars
Best Reply Awards
2
repos

552 contributions in the last year

Pinned
⚡ Create disk images for VMs without root or sudo
⚡ A test go library for learning go mod versioning
Activity
May
19
4 days ago
Activity icon
issue

uablrek issue comment xdp-project/xdp-tutorial

uablrek
uablrek

advanced03-AF_XDP: load file fails on recent Linux kernels

Some time around linux-5.13 loading fails with;

# ./af_xdp_user -d eth1 --filename af_xdp_kern.o
libbpf: Netlink-based XDP prog detected, please unload it in order to launch AF_XDP prog
ERROR: Can't setup AF_XDP socket "Invalid argument"
uablrek
uablrek

Start without --filename works;

# ./af_xdp_user -d eth1
AF_XDP RX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000275
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.000275

AF_XDP RX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.003799
       TX:             0 pkts (         0 pps)           0 Kbytes (     0 Mbits/s) period:2.003799
Activity icon
issue

uablrek issue xdp-project/xdp-tutorial

uablrek
uablrek

advanced03-AF_XDP: load file fails on recent Linux kernels

Some time around linux-5.13 loading fails with;

# ./af_xdp_user -d eth1 --filename af_xdp_kern.o
libbpf: Netlink-based XDP prog detected, please unload it in order to launch AF_XDP prog
ERROR: Can't setup AF_XDP socket "Invalid argument"
Activity icon
issue

uablrek issue comment xdp-project/xdp-tutorial

uablrek
uablrek

Problem for Running XDP on Virtio network device

Hi Dear Fellows: When I do AF_XDP test on Virtio network device, I got a problem as below. So pls help to figure out this problem, I don't know so much regarding virtual network device.

Exception information: libbpf: can't get next link: Invalid argument libbpf: Kernel error message: virtio_net: Too few free TX rings available

Regarding linux sorce code can be found at [email protected] virtnet_xdp_set() function: static int virtnet_xdp_set(struct net_device *dev, struct bpf_prog *prog, struct netlink_ext_ack *extack) { ... ... / XDP requires extra queues for XDP_TX / if (curr_qp + xdp_qp > vi->max_queue_pairs) { NL_SET_ERR_MSG_MOD(extack, "Too few free TX rings available"); netdev_warn(dev, "request %i queues but max is %i\n", curr_qp + xdp_qp, vi->max_queue_pairs); return -ENOMEM; } ... ... }

uablrek
uablrek

(assuming kvm/qemu) You must tell qemu to use more queues. Below is a script function I use to create parametes to kvm. You can't use it directly, but use it as a guideline;

# Mqueue setup. Needed for XDP with 0-copy
netY() {
        local nodeid=$1
        local n=$2
        local tap b0 b1 dev tmp

        dev=xcbr$n
        tap=${dev}_t$nodeid

        tmp=/tmp/tmp/xcnet
        mkdir -p $tmp
        cat > $tmp/$tap <<EOF
#! /bin/sh
echo $tap \$1 > $tmp/$tap.log
ip tuntap add \$1 mode tap user $USER
ip link set dev \$1 master $dev
ip link set up \$1
EOF
        chmod a+x $tmp/$tap
        
        b0=$(printf '%02x' $nodeid)
        b1=$n
        echo " -netdev tap,ifname=$tap,id=net$n,script=$tmp/$tap,queues=4,vhost=on"
        echo " -device virtio-net-pci,mq=on,vectors=6,netdev=net$n,mac=00:00:00:01:0$b1:$b0"
}

NOTE scrip= must be used (for some reason), so a script is generated.

Then in your VM you should see;

vm-001 ~ # ethtool -l eth1
Channel parameters for eth1:
Pre-set maximums:
RX:             n/a
TX:             n/a
Other:          n/a
Combined:       4
Current hardware settings:
RX:             n/a
TX:             n/a
Other:          n/a
Combined:       2
vm-001 ~ # ethtool -L eth1 combined 1
vm-001 ~ # ethtool -l eth1
Channel parameters for eth1:
Pre-set maximums:
RX:             n/a
TX:             n/a
Other:          n/a
Combined:       4
Current hardware settings:
RX:             n/a
TX:             n/a
Other:          n/a
Combined:       1

The ethtool -L eth1 combined 1 is to use just one AF_XDP socket.

(Earlier the "n/a" was "0". Don't know when that changed.)

May
17
6 days ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

ovl/xdp; restructure src/ dir

commit sha: 6ab8013c3de1bda0e6d7f860f3ac8b81e311aae7

push time in 6 days ago
May
16
1 week ago
Activity icon
delete

uablrek in Nordix/Meridio delete branch port-nat

deleted time in 6 days ago
pull request

uablrek pull request Nordix/Meridio

uablrek
uablrek

Add port NAT

A port-NAT is requested to allow un-privileged ports in application pods while exposing priviledged, like 80 -> 8080 for http.

Note to reviewers

The nft operations in a transaction (before nftables.Flush()) are atomic. Meaning that an updated address set can safely be flushed before adding the updated addresses. There is no need for a clumsy and error-prone add-new-remove-leftovers procedure.

push

uablrek push Nordix/xcluster

uablrek
uablrek

Added config/linux-5.17.8. Requires Ubuntu 22

commit sha: 32e31b81fb84dbb0896830356ef80875a9b9d19f

push time in 1 week ago
May
15
1 week ago
Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

fix the pkg/proxy/ipvs/proxier.go can not check the config from a statically compiled kernel

…tically compiled kernel

What type of PR is this?

/kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #108579

Special notes for your reviewer:

Does this PR introduce a user-facing change?


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


uablrek
uablrek

It is a method to check the ipvs, but it's not a good way.

I agree, that's why I did not approve it.

My proposal is here `https://github.com/kubernetes/kubernetes/issues/108579#issuecomment-1125675626.

May
14
1 week ago
started
started time in 1 week ago
started
started time in 1 week ago
started
started time in 1 week ago
started
started time in 1 week ago
started
started time in 1 week ago
Activity icon
wiki
created time in 1 week ago
Activity icon
published release Ubuntu 22.04 base

uablrek in Nordix/xcluster create published release Ubuntu 22.04 base

createdAt 1 week ago
Activity icon
created tag

uablrek in Nordix/xcluster create tag 7.0.0

createdAt 1 week ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

Doc updates for Ubuntu 22.04

commit sha: 68fc2395942ff45aa7df9fc10c2fdd00ddd8ccd5

push time in 1 week ago
May
13
1 week ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

ovl/dpdk; refresh. Use version 21.11.1.

commit sha: a4f34482397cffe16b31fb7bcbf630b4b421727f

push time in 1 week ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

ovl/k8s-cni-calico; upgrade to calico/cni:v3.23.0

commit sha: ec5e8214a30c6c6d0346e544d408259935874ba8

push time in 1 week ago
Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

update for APIs removed in 1.25

Now updated for a state that is possible to merge. I'll likely pick the test-cmd solution from the PSP admission removal.

/kind cleanup

NONE
uablrek
uablrek

This PR breaks functionality for calico/cni:v3.22.1 which works in K8s v1.24.0. Calico 3.23.0 works according to the PR above. While I am aware that this is not a bug, IMHO you should include a warning in the release notes (not "NONE"), unless that is done elsewhere of course.

push

uablrek push Nordix/kubernetes

uablrek
uablrek

fixing the panic in TestVersion

uablrek
uablrek

Fixed portName validation error message.

uablrek
uablrek

change to use require.NoError

uablrek
uablrek

kubelet/stats: update cadvisor stats provider with new log location

in https://github.com/kubernetes/kubernetes/pull/74441, the namespace and name were added to the pod log location.

However, cAdvisor stats provider wasn't correspondingly updated.

since CRI-O uses cAdvisor stats provider by default, despite being a CRI implementation, eviction with ephemeral storage and container logs doesn't work as expected, until now!

Signed-off-by: Peter Hunt [email protected]

uablrek
uablrek

kubelet/stats: take container log stats into account when checking ephemeral stats

this commit updates checkEphemeralStorage to be able to add container log stats, if applicable.

It also updates the old check when container log stats aren't found to be more accurate. Specifically, this check previously worked because of a fluke programming accident:

according to this block in pkg/kubelet/stats/helper.go:113

if result.Rootfs != nil {
    rootfsUsage := *cfs.BaseUsageBytes
    result.Rootfs.UsedBytes = &rootfsUsage
}

BaseUsageBytes should be the value added, not TotalUsageBytes. However, since in this case one also needs to account for the calculated log size, which is TotalUsageBytes - BaseUsageBytes using TotalUsageBytes value accidentally worked.

Updating the case to use the correct value AND log offset fixes this accident and makes the behavior more in line with what happens when calculating ephemeral storage.

Signed-off-by: Peter Hunt [email protected]

uablrek
uablrek

kubelet/stats: add unit test for when container logs are found

Signed-off-by: Peter Hunt [email protected]

uablrek
uablrek

fix typo for nodelifecycle controller

uablrek
uablrek

Modify function parameters

Signed-off-by: xin.li [email protected]

uablrek
uablrek

add verify-licenses.sh hack script (kubernetes#108942)

uablrek
uablrek

update verify-licenses.sh to make it execute in a different git worktree

uablrek
uablrek

Copy recordPluginMetrics in CycleState.Clone

uablrek
uablrek

[e2e][azure] Make internalStaticIP flexible Now, internalStaticIP is hard-coded to "10.240.11.11". Such IP works for aks-engine cluster but not for CAPZ ones (node-subnet 10.1.0.0/16)

Signed-off-by: Zhecheng Li [email protected]

uablrek
uablrek

Promote e2e job lifecycle test to Conformance

uablrek
uablrek

Clarify ExternalTrafficPolicy/InternalTrafficPolicy definitions

uablrek
uablrek

Drop unused golang/template package

uablrek
uablrek

Drop unused golang/template funcs

uablrek
uablrek
uablrek
uablrek

Update Metrics doc as there is a typo in package

Package header typo is very visible looking at docs.

https://pkg.go.dev/k8s.io/metrics/pkg/apis/metrics

uablrek
uablrek

Optimize test cases for ipvs

uablrek
uablrek

Optimize test cases for iptables

commit sha: 9d85e18ec0dc09f681650d04d65e3db5237bef96

push time in 1 week ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

ovl/k8s-*; removed obsolete single-stack doc

commit sha: 7dfa9d2a87a6e7667c177dbb045c259877795209

push time in 1 week ago
push

uablrek push Nordix/xcluster

uablrek
uablrek

ovl/test-template; remove single-stack tests

commit sha: a10a13ea6a1a94700fd746e334ce574c84068f29

push time in 1 week ago
Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

Add a new method to check ipvs modules via proc-fs

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add a new method to check ipvs modules via proc-fs. It is not necessary to check each module, to a large extent, ipvs's modules are built entirely and can load its own module automatically. Anyway it is hard to cover every environment, for example kernel and modules are built-in, but no kernel config file, like ipvs proxyer runs in a container. Check-up on running environment is better than check-up on specific files, it may introduce new dependency. So this patch take proc-fs into consideration.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer: Does this PR introduce a user-facing change? No

Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

The code proxier.go failed to check kernel module when the folder /proc/module is not exsit

What happened?

When the kernel is statically compiled and statically loaded, and the folder /proc/modules is not exsit. In the kubernetes, use the ipvs. The code proxier.go failed to check /proc/module when it is not exsit. In the line 628, the err is not nil. But when the folder /proc/modules is exsit, it go to the line 639. The question is that, how the code runs to 614 line?

What did you expect to happen?

In the kubernetes, use the ipvs. the kernel is statically compiled and statically loaded. It check out the config in the folder /boot, such as /boot/config-****

How can we reproduce it (as minimally and precisely as possible)?

If you choose an arm64 machine, config the kernel config , set CONFIG_MODULES=no, compile statically the kernel. Join the node into k8s's cluster. So you will see this in the pod kube-proxy***.

Anything else we need to know?

No response

Kubernetes version

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.16", GitCommit:"e37e4ab4cc8dcda84f1344dda47a97bb1927d074", GitTreeState:"clean", BuildDate:"2021-10-27T16:25:59Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19", GitVersion:"v1.19.16", GitCommit:"e37e4ab4cc8dcda84f1344dda47a97bb1927d074", GitTreeState:"clean", BuildDate:"2021-10-27T16:20:18Z", GoVersion:"go1.15.15", Compiler:"gc", Platform:"linux/amd64"}

Cloud provider

There is not a cloud provider. This is for myself.

OS version

# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux armk1101 4.9.216-q4_2107u2

Install tools

Container runtime (CRI) and version (if applicable)

Related plugins (CNI, CSI, ...) and versions (if applicable)

uablrek
uablrek

TL;DR

I propose;

  • Ditch the module checks entirely
  • Check the ipvs function by doing a list command (will automativally load the module if needed)
  • Assume that the rr (round-robin) scheduler exist if ipvs is supported (no explicit check)
  • Check the conntrack function by doing a list command (will automativally load the module if needed)

@thockin @andrewsykim @aojea Please comment on the above.

Background

Now we have 2 PR's that adds ways to check modules; https://github.com/kubernetes/kubernetes/pull/110017 and https://github.com/kubernetes/kubernetes/pull/109937.

As mentioned in https://github.com/kubernetes/kubernetes/pull/109937#issuecomment-1123218857 I would like to remove the explicit module check entirely, rather than put another band-aid on a flawed function.

To check the function a good way is to make a non-intrusive ipvs operation (not calling any external program), for instance;

https://github.com/kubernetes/kubernetes/blob/b74d023e70d6064c7f3f77031e7d26ec38497fc9/pkg/util/ipvs/ipvs.go#L39

If that succeeds, the ipvs function can be used (proxy-mode=ipvs can be used).

The current function also checks the rr, wrr and sh ipvs schedulers. Only rr is used though, other schedulers can be configured explicitly, but IMO they are not needed for the function and must not be checked. IMHO it is safe to assume that if ipvs is in the kernel (statically or by module) then the rr sheduler also exist.

The conntracker module is also checked. Again the conntrack function can be checked with some non-intrusive command, like a list command.

Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

Add a new method to check ipvs modules via proc-fs

What type of PR is this?

/kind feature

What this PR does / why we need it:

Add a new method to check ipvs modules via proc-fs. It is not necessary to check each module, to a large extent, ipvs's modules are built entirely and can load its own module automatically. Anyway it is hard to cover every environment, for example kernel and modules are built-in, but no kernel config file, like ipvs proxyer runs in a container. Check-up on running environment is better than check-up on specific files, it may introduce new dependency. So this patch take proc-fs into consideration.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer: Does this PR introduce a user-facing change? No

Activity icon
issue

uablrek issue comment kubernetes/kubernetes

uablrek
uablrek

fix the pkg/proxy/ipvs/proxier.go can not check the config from a statically compiled kernel

…tically compiled kernel

What type of PR is this?

/kind bug

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #108579

Special notes for your reviewer:

Does this PR introduce a user-facing change?


Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


May
12
1 week ago
Activity icon
delete

uablrek in Nordix/Meridio delete branch proxy-err-logging

deleted time in 1 week ago
push

uablrek push Nordix/Meridio

uablrek
uablrek

Fix ERRO printouts in the proxy logs

commit sha: 121ea9f3b5a01aa71d48f180c889c8f36a0d8239

push time in 1 week ago
Previous