03 Jan 2019, 22:11

Kube apiserver Audit Logs and jq

In an attempt to save these somewhere, this is just a collection of some useful one liners for pulling information out of the Kubernetes apiserver audit logs using jq.

  • Show only completed responses:

    Requests and responses are tracked slightly separately. A separate event is logged for each, though the response event log includes the request information.

    gzcat $AUDITLOG | jq 'select(.stage="ResponseComplete")
  • Show all completed actions by a specific user:

    ... | jq 'select(.user.username=="$NAME")

    $NAME can be an end user or a fully qualified service account (system:serviceaccount:namespace:saname).

  • Ordered count of requests by user:

    This might take a bit.

    gzcat $AUDITLOG | jq 'select(.stage="ResponseComplete") | .user.username' | sort | uniq -c
  • Ignore all requests by a specific user:

    ... | jq 'select(.user.username!="$NAME")
  • Show only the first and last entry:

    Since the entries tend to be verbose and it’s easy to get lost with lots of lines scrolling past, it’s useful to trim down to the first few entries (and it’s more complete than | head -??)

    ... | jq -s 'first,last'

    Note that this one might use too much memory or take a while since it has to read everything in (even for just first) before it can process it. If you’re just doing first or last, you can probably do a | head -100 or | tail -100 before |jq and that can speed it up.

  • Remove some boilerplate fields:

    Most of the time you’re not going to use these.

    ... | jq 'del(.kind) | del(.apiVersion) | del(.metadata) | del(.level)
  • Show rate throttled requests:

    In case your apiserver is getting overwhelmed, and you want to find from where.

    ... | jq '.responseStatus.code==429'