bowei

bowei issue comment kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Do not include User Errors in L4NetLBInError metric

There are errors related to Network Tier that are caused by User misconfigurations and they should be filtered out from ServicesInError metrics. This errors are:

Ip address and forwarding rule Network Tier mismatch
User created service in Standard Tier and deleted the annotation, this action will changed desired service Network Tier to PREMIUM but IP address and forwarding rule should be tear down first because we cannot update Network Tier in this resources. We can tear down resources only when Network Tier from annotation mismatch with network tier in IP address or forwarding rule.

bowei

/lgtm /approve

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

Don't we need to take the lock here to avoid something creating in parallel to the deletion? This has a lock read unlock; A ; lock write unlock issue -- there might be some operation occurring at A that is a race.

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

can we use this comment?

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

you don't need this if any more

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

You need to put this first, no? the instance != nil needs to be guarded.

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

We should be more explicit:

// sharedResourceLock serializes operations on the healthcheck and firewall // resources shared across multiple Services.

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

go style puts the mutex in the same block as the protected var:

struct {
  lock
  var
  var

  nonProtectedVar
  ...
}

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

it's easier to maintain if we make this straight line code:

if ! utils.EqualStringSets() { return false }
if ! utils.EqualStringSets() { return false }
if ! skipDescription && a.Description != b.Description {
  return false
}
return true

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

can we add a comment describing why

bowei

bowei CONTRIBUTOR

Rewrite L4 healthchecks creation and deletion

create a common singleton like struct fot l4 health checks
new struct holds mutex for common resources
delete shared healtcheck firewall rules safely

without proper guarding of shared healtcheck firewall rules, the NetLB controller may delete the ILB healthcheck firewall or vice versa

bowei

bowei MEMBER

May

6 days ago

bowei issue comment kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Truncate nodes list to maximum instance group size (1000) before adding to instance group

This fixes a bug for clusters bigger than instance group max size, cause we were trying to add more than allowed number of nodes to instance group and gcp api returned an error

bowei

/approve

Ping back when you add the comment.

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Truncate nodes list to maximum instance group size (1000) before adding to instance group

This fixes a bug for clusters bigger than instance group max size, cause we were trying to add more than allowed number of nodes to instance group and gcp api returned an error

bowei

Let's add a comment here so future reviewers don't have the same question

// List() will return a sorted list so the kubeNodesList truncation will have a stable set of nodes.

bowei

bowei CONTRIBUTOR

Truncate nodes list to maximum instance group size (1000) before adding to instance group

This fixes a bug for clusters bigger than instance group max size, cause we were trying to add more than allowed number of nodes to instance group and gcp api returned an error

bowei

bowei MEMBER

bowei issue comment kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Truncate nodes list to maximum instance group size (1000) before adding to instance group

This fixes a bug for clusters bigger than instance group max size, cause we were trying to add more than allowed number of nodes to instance group and gcp api returned an error

bowei

/ok-to-test

bowei issue comment kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

/lgtm /approve

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

We can do this as a follow up

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

Can we have this at V(2) so it appears in the normal log? Is it too noisy?

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

bowei MEMBER

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Fix for backend service update

We should update backend service with ig llinks only when ig links changed.

bowei

Same comment here V(2)

bowei

bowei CONTRIBUTOR

Do not include User Errors in L4NetLBInError metric

There are errors related to Network Tier that are caused by User misconfigurations and they should be filtered out from ServicesInError metrics. This errors are:

Ip address and forwarding rule Network Tier mismatch
User created service in Standard Tier and deleted the annotation, this action will changed desired service Network Tier to PREMIUM but IP address and forwarding rule should be tear down first because we cannot update Network Tier in this resources. We can tear down resources only when Network Tier from annotation mismatch with network tier in IP address or forwarding rule.

bowei

bowei MEMBER

submittedAt 6 days ago

minor comments

bowei wants to merge kubernetes/ingress-gce

bowei

bowei CONTRIBUTOR

Do not include User Errors in L4NetLBInError metric

There are errors related to Network Tier that are caused by User misconfigurations and they should be filtered out from ServicesInError metrics. This errors are:

Ip address and forwarding rule Network Tier mismatch
User created service in Standard Tier and deleted the annotation, this action will changed desired service Network Tier to PREMIUM but IP address and forwarding rule should be tear down first because we cannot update Network Tier in this resources. We can tear down resources only when Network Tier from annotation mismatch with network tier in IP address or forwarding rule.

bowei

comment in 6 days ago

I think you can just do

var ( networkTierErrorRegexp = regexp.MustCompile...