limit number of updates to fqdnstate configmap

[mwennrich]

Description

Updating the fqdnstate configmap on every dns-request puts a lot of stress on the kube-apiserver/etcd on busy clusters:

kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.007434       1 timeout.go:140] "Post-timeout activity" logger="UnhandledError" timeElapsed="4.451779ms" method="PUT" path="/api/v1/namespaces/firewall/configmaps/fqdnstate" result=null
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.201733       1 finisher.go:175] "Unhandled Error" err="FinishRequest: post-timeout activity - time-elapsed: 5.501µs, panicked: false, err: context canceled, panic-reason: <nil>" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.201771       1 writers.go:123] "Unhandled Error" err="apiserver was unable to write a JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.202965       1 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"http: Handler timeout\"}: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.204130       1 writers.go:136] "Unhandled Error" err="apiserver was unable to write a fallback JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.206016       1 timeout.go:140] "Post-timeout activity" logger="UnhandledError" timeElapsed="4.342528ms" method="PUT" path="/api/v1/namespaces/firewall/configmaps/fqdnstate" result=null
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.603290       1 wrap.go:53] "Timeout or abort while handling" logger="UnhandledError" method="PUT" URI="/api/v1/namespaces/firewall/configmaps/fqdnstate" auditID="dc266472-5411-4047-bb70-d7a0fadbac4a"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:34.603772       1 timeout.go:140] "Post-timeout activity" logger="UnhandledError" timeElapsed="441.437µs" method="PUT" path="/api/v1/namespaces/firewall/configmaps/fqdnstate" result=null
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:35.600925       1 writers.go:123] "Unhandled Error" err="apiserver was unable to write a JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:35.600921       1 finisher.go:175] "Unhandled Error" err="FinishRequest: post-timeout activity - time-elapsed: 6.34µs, panicked: false, err: context canceled, panic-reason: <nil>" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:35.602071       1 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"http: Handler timeout\"}: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:35.603241       1 writers.go:136] "Unhandled Error" err="apiserver was unable to write a fallback JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:35.605062       1 timeout.go:140] "Post-timeout activity" logger="UnhandledError" timeElapsed="4.23787ms" method="PUT" path="/api/v1/namespaces/firewall/configmaps/fqdnstate" result=null
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:36.203236       1 writers.go:123] "Unhandled Error" err="apiserver was unable to write a JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:36.203304       1 finisher.go:175] "Unhandled Error" err="FinishRequest: post-timeout activity - time-elapsed: 127.629µs, panicked: false, err: Operation cannot be fulfilled on configmaps \"fqdnstate\": the object has been modified; please apply your changes to the latest version and try again, panic-reason: <nil>" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:36.204452       1 status.go:71] "Unhandled Error" err="apiserver received an error that is not an metav1.Status: &errors.errorString{s:\"http: Handler timeout\"}: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:36.205686       1 writers.go:136] "Unhandled Error" err="apiserver was unable to write a fallback JSON response: http: Handler timeout" logger="UnhandledError"
kube-apiserver-7b5bccd87d-lz86c kube-apiserver E0416 06:37:36.207859       1 timeout.go:140] "Post-timeout activity" logger="UnhandledError" timeElapsed="4.711571ms" method="PUT" path="/api/v1/namespaces/firewall/configmaps/fqdnstate" result=null

With this PR, we update the fqdnstate configmap only every 10 seconds.