Since the inception of Falco, we’ve seen users write custom rules covering a number of different use cases. Because Falco is behavioral monitoring with a syntax that leverages system calls, you can write a rule for just about anything: opening a file, becoming root, or making a network connection.
Today I’m going to talk about using Falco in a more traditional manner – using a rule to detect unexpected network activity. A Sysdig Secure customer wanted a policy to trigger if a connection was established to an unknown (and thus untrusted) domain names.
Out of the box Falco has a macro to detect outbound network activity:
- macro: outbound
condition: >
(((evt.type = connect and evt.dir=<)) or
(fd.typechar = 4 or fd.typechar = 6) and
(fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8") and
(evt.rawres >= 0 or evt.res = EINPROGRESS))
Let’s break this down:evt.type = connect and evt.dir=< is looking for a “connect” exit event
fd.typechar = 4 or fd.typechar = 6 is matching a IPv4 or v6 file descriptor
fd.ip != "0.0.0.0" and fd.net != "127.0.0.0/8" is excluding local connections
evt.rawres >= 0 or evt.res = EINPROGRESS ensures we only look for successful (or in progress) socket binds
With this macro we can easily write a rule matching against a connection to an IP address. This rule will trigger upon establishing connection with sysdig.com:
$ host sysdig.com
sysdig.com has address 35.184.21.208
- rule: Connection to sysdig.com
desc: Detect attempts to connect to sysdig.com (35.184.21.208)
condition: outbound and fd.sip="35.184.21.208"
output: Outbound connection to sysdig.com (command=%proc.cmdline connection=%fd.name %container.info image=%container.image)
priority: NOTICE
tags: [network]
This gets us part way there, but what if sysdig.com has dynamic DNS resulting in the IP address changing? Our rule would be useless, as we’ve hardcoded in a static IP 35.184.21.208. For example, s3.us-east-2.amazonaws.com can resolve to different IP addresses on each lookup:
$ host s3.us-east-2.amazonaws.com
s3.us-east-2.amazonaws.com has address 52.219.96.58
$ host s3.us-east-2.amazonaws.com
s3.us-east-2.amazonaws.com has address 52.219.104.82
We could keep a number of different IP addresses in a Falco list:
- list: s3_ips
items: ['"52.219.96.58"', '"52.219.104.82"']
- rule: Connection to S3
desc: Detect attempts to connect to S3
condition: outbound and fd.sip in (s3_ips)
output: Outbound connection to S3 URL(command=%proc.cmdline connection=%fd.name %container.info image=%container.image)
priority: NOTICE
tags: [network]
… but it wouldn’t be easy to ensure this list is fresh and up to date, resulting in expected network connections to trigger a policy.
Hostnames on network rules using fd.sip.name
As a member of our customer success team, I’m well aware how receptive Sysdig’s product teams are to customer requests. Realising the awkwardness of dynamic DNS and (not wanting) hard coded IP addresses within Falco rules, our engineering team worked on a better way to solve this problem. In version 0.24.0 of Sysdig open source, we added a number of new filter checks:
fd.cip.name Domain name associated with the client IP address.
fd.sip.name Domain name associated with the server IP address.
fd.lip.name Domain name associated with the local IP address.
fd.rip.name Domain name associated with the remote IP address.
These new filterchecks allow specifying a domain name, and Sysdig will resolve the IP address, intelligently maintaining the IP addresses it resolves to. This means we can now easily pass Falco domain names, and not have to worry about unreliably resolving the IP addresses and keeping them up to date.
In the example below, I have defined a list of trusted domain names (sysdig.com, github.com & google.com). Any network connection to an IP address that isn’t resolved to by any of these domain names will trigger the policy. Great for telling Falco what’s allowed and being told about everything else.
- list: trusted_domainsUsing DNS based network rules to control inbound and outbound traffic from your #containers in #Kubernetes Click to tweet
items: [sysdig.com, github.com, google.com]
- rule: Unexpected outbound network connection
desc: Detect outbound connections with destinations not on allowed list
condition: >
outbound
and not (fd.sip.name in (trusted_domains))
output: Unexpected Outbound Connection
(container=%container.name
command=%proc.cmdline
procpname=%proc.pname
connection=%fd.name
servername=%fd.sip.name
serverip=%fd.sip
type=%fd.type
typechar=%fd.typechar
fdlocal=%fd.lip
fdremote=%fd.rip)
priority: NOTICE
Dynamic DNS based network policy for Kubernetes
Once a Falco rule has been written it can be easily added to Sysdig Secure. Rules can be associated with a policy, scoped to a specific portion of your infrastructure, and tied to remediation actions like killing or pausing a container.
In the example below the above Falco rule has been applied to the scope of kubernetes.deployment.name = javaapp meaning if any container with the metadata in that scoping makes a connection to an IP address outside the IPs our domains resolve to, then an alert will be fired and in this case the container will also be killed.
Using the labels from Kubernetes or your cloud provider can be a powerful way to make sure your policies are tailored to your specific clusters, namespaces, or VPC’s. Sysdig Secure does all of this tagging automatically and manages the distribution of Falco rules to all your endpoints. Even if they’re across thousands of nodes.
Conclusion
Hopefully this blog gives you an idea of how powerful system calls can be as used a data source for intrusion detection, auditing, and behavioral monitoring. We’d love to hear more from you about what you’re doing with Falco in the wild. Reach out to us on Twitter or join our Slack.