Falco – Kubernetes

Blog: Falco at KubeCon Europe 2026 — See You in Amsterdam! 🐦

Wed, 18 Mar 2026 00:00:00 +0000

We're excited to share that the Falco community will be at KubeCon + CloudNativeCon Europe 2026 in Amsterdam! Whether you're a long-time contributor, a curious user, or just want to say hi, we'd love to see you there.

Falco is celebrating 10 years of development and adoption, and we are on the lookout for people who would like to say Happy Birthday to the project or share their best Falco story. Libby Schulze and I will be on the event floor with mic and camera to capture some amazing moments and memories from Falco's 10 years. So bring your best story, and we'll see you at the Falco booth!

Sneak peek

Psst... we have something really cool brewing that we will show at the Falco booth. You, our amazing reader, is the first to hear about this. It's a way to run Falco locally on your development machine, and make sure your AI coding agents are following new rules that are being defined. We'd love to get your feedback on this as we're currently building it!

Here’s where you can find us in Amsterdam and everything we have lined up:

Project lightning talk

Forensics With Falco
Speaker: Gerald Combs, Maintainer
When: Monday, March 23, 2026 — 10:27 to 10:32 CET
Where: Elicium 2

Falco has recently expanded its capabilities with capture recording, opening the door to seamless integration with forensic analysis tools like Stratoshark. In this lightning talk, Gerald will walk through how the two tools work together to provide deep visibility into container and system activity. He will demonstrate how captured event data can accelerate investigations and discuss key considerations for safely and efficiently deploying these features in production environments.

Sysdig-led workshop

Hands-On Cloud Native Security Workshop
When: Monday, March 23 — 2:00–4:00 PM CET

Run Atomic Red Team™ tests, then step into the Blue Team role to detect threats and create custom Falco™ detection rules in this hands‑on 90‑minute keyboard workshop.

Conference talk

In Falco's Nest: The Evolution of Cloud Native Runtime Security
Speakers: Iacopo Rozzo (Sysdig), Aldo Lacuku (Kong Inc.)
When: Tuesday, March 24, 2026 — 12:00 to 12:30 CET
Where: G102–103

Falco, the Cloud Native Runtime Security project, is constantly evolving to meet the demands of modern cloud environments. This maintainer track session, led by the Falco maintainers, will dive deep into the latest advancements and the strategic direction of the project. We will focus on two major areas of growth: the introduction of the new Falco Operator and the new features that enhance Falco's performance and reliability.

The new Falco Operator simplifies the deployment, configuration, and management of Falco across Kubernetes clusters, making it easier than ever for users to secure their runtime environments at scale.

Furthermore, we will explore the most significant new features integrated into Falco. This includes performance optimizations for high-throughput environments. The session will also touch upon community contributions, ecosystem integrations, and the roadmap for the upcoming release.

Booth demo

Pivoting from detection to investigation with Falco and Stratoshark
Speaker: Gerald Combs
When: Tuesday, March 24, 2026 — 15:45 CET
Where: Sysdig Booth #671

See how to move from “we detected something” to “here’s what happened” using Falco and Stratoshark. Stop by the Sysdig booth and say hello!

Thank you!

We couldn’t do this without you all in our community - the contributors, users, and everyone who shows up at events. If you’re in Amsterdam, come find us at the talks, the workshop, or the booth. We’d love to meet you and hear how you’re using Falco.

See you there! 🐦

Blog: Detecting Supply Chain Attacks with Falco Actions

Wed, 19 Mar 2025 00:00:00 +0000

The recently discovered CVE for the GitHub action tj-actions/changed-files brought to light a topic that is really critical for companies: supply chain attacks. With that, we want to discuss and show a bit about how Falco can help your organization detect this kind of attack and other suspect behaviors inside your CI/CD pipeline.

What is Falco?

Falco is a cloud native security tool that provides runtime security across hosts, containers, Kubernetes, and cloud environments. It leverages custom rules on Linux kernel events and other data sources through plugins, enriching event data with contextual metadata to deliver real-time alerts. Falco enables the detection of abnormal behavior, potential security threats, and compliance violations.

What is Falco Actions?

Falco Actions enable you to run Falco in GitHub Actions to detect suspicious behavior in your CI/CD workflows. If you run it in a pull request, the action will create a comment with the findings.

Thanks to ad-hoc Falco rules specific to this use case, these GitHub actions can monitor your GitHub runner and detect software supply chain attacks.

Using Falco Actions

To have Falco inside your pipeline, you need to add these two actions:

falcosecurity/falco-actions/start
falcosecurity/falco-actions/stop

Below you can see an example:

name: CI

on:
 push:
 pull_request:

jobs:
 build:
 runs-on: ubuntu-latest
 permissions:
 contents: read
 actions: read

 steps:
 - uses: actions/checkout@v4

 - name: Start Falco
 uses: falcosecurity/falco-actions/start@main
 with:
 mode: live
 falco-version: '0.40.0'
 verbose: true

 - name: My Custom Step
 run: |
 echo "This is my custom step"

 - name: Stop Falco
 uses: falcosecurity/falco-actions/start@main
 with:
 mode: live
 verbose: true

OBS: main is being used here only to simplify how it works, you should always pin your dependencies to a specific commit SHA.

After the execution, you will be able to see the results at the github action summary.

If you want a more detailed report, you can use the action falcosecurity/falco-actions/analyze; it will allow you to have a better report with information like:

Falco rules triggered during steps' execution.
Contacted IPs
Contacted DNS domains
SHA256 hash of spawned executables
Spawned container images
Written files
A summary of the report generated with OpenAI
Reputation of Contacted IPs
Reputation of SHA256 hashes

For more informations about the usage, you can check the github repository for the actions.

Default rules file

By default, Falco action will detect a variety of events, following the default CICD rules, that can be overridden if you want.

In the example from the tj-actions/changed-files exploit, one rule that would be triggered is the Process Dumping Memory of Others, which was used during the exploit to dump environment variables from the main process and print them as part of the Github runner execution.

The Falco team is always adding new rules to ensure our users get value out of the box, but you can also write your own rules according to your company policy.

Conclusions

These actions are just the beginning of having the Falco capabilities inside the CI/CD pipelines. You can customize and have your own set of rules, keeping all environments and scenarios covered and protected from supply chain attacks.

Let's meet!

As always, we meet every 2 weeks on Wednesday at 4pm UTC in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Enjoy 😎,

Igor and Edson

Blog: Detecting Threats in OVHcloud MKS Audit Logs with Falco

Thu, 13 Mar 2025 00:00:00 +0000

Detecting threats in a Kubernetes cluster can be challenging, we generally don't know where and how to start. The good news is that we have an amount of valuable logs that can help us to know what is happened in the cluster. Indeed, each action requested or done by a user or an app, in a cluster, is recorded in Audit Logs. Kubernetes events are key to understanding the behavior of a cluster.

We already provide plugins that let you parse Audit Logs and use Falco to detect threats from GKE, EKS and AKS clusters. With our latest plugin, you'll now have the same powerful threat detection capabilities for your OVHcloud MKS clusters 🎉.

What is Falco?

Falco is an Open Source cloud-native runtime security tool. It provides near real-time threat detection for cloud, container, and Kubernetes workloads by leveraging runtime insights. Falco can monitor events from various sources, including the Linux kernel, and enrich them with metadata from the Kubernetes API server, container runtime, and more.

Falco can receive Events, compare them to a set of Rules to determine the actions to perform and generate Alerts to different endpoints.

What is the OVH MKS Audit Logs plugin?

The OVH audit logs plugin (k8saudit-ovh) extends Falco's capabilities to OVHcloud Managed Kubernetes Service (MKS) clusters, providing you with the same security insights and threat detection Falco already offers for GKE, EKS and AKS environments.

With this plugin, you can seamlessly integrate MKS Audit Logs into Falco's event processing pipeline, enabling it to identify anomalies, suspicious activities, and policy violations within your MKS-based workloads.

Concretely, when a user executes some kubectl commands in an OVHcloud MKS cluster, Audit Logs will be generated. Falco is listening to them, and depending on the configured rules to follow, it will generate some alerts.

Using OVH MKS Audit Logs plugin

In order to use the OVH MKS Audit Logs plugin, you must follow several steps:

deploy an OVHcloud LDP (Logs Data Platform)
create a data stream into this LDP
connect an OVHcloud MKS cluster to the data stream (to send Audit Logs into it)

To be able to access our Kubernetes clusters' Audit Logs, you need to deploy an LDP. LDP is the managed platform for collecting, processing, analyzing, and storing your logs of the OVHcloud products. Deploy an LDP (Bare Metal Cloud universe) with whatever plan you want.

OVHcloud Kubernetes Audit Logs will be stored in a data stream. The OVHcloud Audit Logs Falco plugin receive the audit logs through Websocket so you need to enable Websocket broadcasting when you create the data stream on LDP.

Retrieve the Websocket URL, follow the guide to do so. The Websocket address have this kind of format: wss://gra.logs.ovh.com/tail/?tk=

Finally, you have to connect a MKS cluster to the LDP data stream.

Configuring Falco to use OVH Audit Logs plugin

Running locally

If you have a Falco running locally, using falcoctl, add the falcosecurity index (if it's not already the case) and install the k8saudit-ovh Falco plugin:

# Add falcosecurity index
sudo falcoctl index add falcosecurity https://falcosecurity.github.io/falcoctl/index.yaml
# Install k8saudit-ovh Falco plugin
sudo falcoctl artifact install k8saudit-ovh

Fill your falco.yaml file in order to add the plugin configuration:

plugins:
 - name: k8saudit-ovh
 library_path: /usr/share/falco/plugins/libk8saudit-ovh.so
 open_params: "<OVH LDP WEBSOCKET URL>" # gra<x>.logs.ovh.com/tail/?tk=<ID>
 - name: json
 library_path: libjson.so
 init_config: ""
load_plugins: [k8saudit-ovh, json]

stdout_output:
 enabled: true

Running in a Kubernetes cluster

If you have a Falco running in a Kubernetes cluster (on OVHcloud MKS or on another cluster), deployed with Helm, create a values.yaml file with the following content:

tty: true
kubernetes: false

# Just a Deployment with 1 replica (instead of a Daemonset) to have only one Pod that pulls the MKS Audit Logs from a OVHcloud LDP
controller:
 kind: deployment
 deployment:
 replicas: 1

falco:
 rule_matching: all
 rules_files:
 - /etc/falco/k8s_audit_rules.yaml
 - /etc/falco/rules.d
 plugins:
 - name: k8saudit-ovh
 library_path: libk8saudit-ovh.so
 open_params: "gra<x>.logs.ovh.com/tail/?tk=<ID>" # Replace with your LDP Websocket URL
 - name: json
 library_path: libjson.so
 init_config: ""
 # Plugins that Falco will load. Note: the same plugins are installed by the falcoctl-artifact-install init container.
 load_plugins: [k8saudit-ovh, json]

driver:
 enabled: false
collectors:
 enabled: false

# use falcoctl to install automatically the plugin and the rules
falcoctl:
 artifact:
 install:
 enabled: true
 follow:
 enabled: true
 config:
 indexes:
 - name: falcosecurity
 url: https://falcosecurity.github.io/falcoctl/index.yaml
 artifact:
 allowedTypes:
 - plugin
 - rulesfile
 install:
 resolveDeps: false
 refs: [k8saudit-rules:0, k8saudit-ovh:0.1, json:0]
 follow:
 refs: [k8saudit-rules:0]

This values.yaml file will install Falco with the k8saudit-ovh and the json plugins.

Install the latest version of Falco with helm install command:

$ helm install falco --create-namespace --namespace falco --values=values.yaml falcosecurity/falco

This command will install the latest version of Falco, with the k8saudit-ovh and json plugins, and create a new falco namespace.

Or if you already have Falco deployed in a Kubernetes cluster, you can use the helm update command instead:

$ helm update falco --create-namespace --namespace falco --values=values.yaml falcosecurity/falco

Once the Falco pod is ready, run the following command to see the logs:

kubectl logs -l app.kubernetes.io/name=falco -n falco -c falco

You should see logs like that:

$ kubectl logs -l app.kubernetes.io/name=falco -n falco -c falco

Mon Feb 10 09:15:35 2025: /etc/falco/k8s_audit_rules.yaml | schema validation: ok
Mon Feb 10 09:15:35 2025: Hostname value has been overridden via environment variable to: my-pool-1-node-921b61
Mon Feb 10 09:15:35 2025: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Mon Feb 10 09:15:35 2025: Starting health webserver with threadiness 2, listening on 0.0.0.0:8765
Mon Feb 10 09:15:35 2025: Loaded event sources: syscall, k8s_audit
Mon Feb 10 09:15:35 2025: Enabled event sources: k8s_audit
Mon Feb 10 09:15:35 2025: Opening 'k8s_audit' source with plugin 'k8saudit-ovh'
{"hostname":"my-pool-1-node-921b61","output":"09:15:40.698757000: Warning K8s Operation performed by user not in allowed list of users (user=csi-cinder-controller target=csi-6afb06dce281b86b7bab718b5d966dc261b2b1554941ae449519a128cb2e3fb3/volumeattachments verb=patch uri=/apis/storage.k8s.io/v1/volumeattachments/csi-6afb06dce281b86b7bab718b5d966dc261b2b1554941ae449519a128cb2e3fb3/status resp=200)","output_fields":{"evt.time":1739178940698757000,"ka.response.code":"200","ka.target.name":"csi-6afb06dce281b86b7bab718b5d966dc261b2b1554941ae449519a128cb2e3fb3","ka.target.resource":"volumeattachments","ka.uri":"/apis/storage.k8s.io/v1/volumeattachments/csi-6afb06dce281b86b7bab718b5d966dc261b2b1554941ae449519a128cb2e3fb3/status","ka.user.name":"csi-cinder-controller","ka.verb":"patch"},"priority":"Warning","rule":"Disallowed K8s User","source":"k8s_audit","tags":["k8s"],"time":"2025-02-10T09:15:40.698757000Z"}
{"hostname":"my-pool-1-node-921b61","output":"09:15:57.508657000: Warning K8s Operation performed by user not in allowed list of users (user=yacht target=my-pool-1.18051c0a88716868/events verb=patch uri=/api/v1/namespaces/default/events/my-pool-1.18051c0a88716868 resp=403)","output_fields":{"evt.time":1739178957508657000,"ka.response.code":"403","ka.target.name":"my-pool-1.18051c0a88716868","ka.target.resource":"events","ka.uri":"/api/v1/namespaces/default/events/my-pool-1.18051c0a88716868","ka.user.name":"yacht","ka.verb":"patch"},"priority":"Warning","rule":"Disallowed K8s User","source":"k8s_audit","tags":["k8s"],"time":"2025-02-10T09:15:57.508657000Z"}
{"hostname":"my-pool-1-node-921b61","output":"09:15:57.807013000: Warning K8s Operation performed by user not in allowed list of users (user=yacht target=my-pool-1/nodepools verb=update uri=/apis/kube.cloud.ovh.com/v1alpha1/nodepools/my-pool-1/status resp=200)","output_fields":{"evt.time":1739178957807013000,"ka.response.code":"200","ka.target.name":"my-pool-1","ka.target.resource":"nodepools","ka.uri":"/apis/kube.cloud.ovh.com/v1alpha1/nodepools/my-pool-1/status","ka.user.name":"yacht","ka.verb":"update"},"priority":"Warning","rule":"Disallowed K8s User","source":"k8s_audit","tags":["k8s"],"time":"2025-02-10T09:15:57.807013000Z"}

Let's test it!

In order to test Falco we need to know which rules are installed by default. In our case, as we defined it in the values.yaml file, the k8saudit-ovh plugin follow the k8s_audit_rules.yaml file. You can take a look at them in order to know them.

In this blog post we will test one of the well-known default k8s audit rules:

- rule: Attach/Exec Pod
 desc: >
 Detect any attempt to attach/exec to a pod
 condition: kevt_started and pod_subresource and (kcreate or kget) and ka.target.subresource in (exec,attach) and not user_known_exec_pod_activities
 output: Attach/Exec to pod (user=%ka.user.name pod=%ka.target.name resource=%ka.target.resource ns=%ka.target.namespace action=%ka.target.subresource command=%ka.uri.param[command])
 priority: NOTICE
 source: k8s_audit
 tags: [k8s]

This rule is interesting because an event will be generated if/when an user execute commands in a pod.

Let’s test the rule!

In a tab of your terminal, watch the coming logs:

$ kubectl logs -l app.kubernetes.io/name=falco -n falco -c falco -f

In an another tab of your terminal, create a Nginx pod and execute a command into it:

$ kubectl run nginx --image=nginx

$ kubectl exec -it nginx -n hello-app -- cat /etc/shadow

Several seconds laters, in the logs you should see this you will see this Attach/Exec to pod logs:

...
{"hostname":"my-pool-1-node-921b61","output":"09:29:46.302906000: Notice Attach/Exec to pod (user=kubernetes-admin pod=nginx-676b6c5bbc-4xc6t resource=pods ns=hello-app action=exec command=cat)","output_fields":{"evt.time":1739179786302906000,"ka.target.name":"nginx-676b6c5bbc-4xc6t","ka.target.namespace":"hello-app","ka.target.resource":"pods","ka.target.subresource":"exec","ka.uri.param[command]":"cat","ka.user.name":"kubernetes-admin"},"priority":"Notice","rule":"Attach/Exec Pod","source":"k8s_audit","tags":["k8s"],"time":"2025-02-10T09:29:46.302906000Z"}
...

💪

Let's meet!

If you have planned to go to the KubeCon + CloudNative Con EU 2025 at London, don't hesitate to stop at the Falco booth in the Project Pavillon!

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Enjoy 😎,

Aurélie

Blog: Falco plugin for collecting AKS audit logs

Sun, 09 Mar 2025 00:00:00 +0000

Troubleshooting Kubernetes events is challenging due to the multitude of data sources involved: container logs, Kubernetes events, cloud logs, and more. Among these sources, Kubernetes audit logs are especially valuable for identifying threats, as every action passing through the Kubernetes API server is recorded there.

We already provide plugins that let you parse and use Falco to detect threats in audit logs from GKE and EKS clusters. With our latest plugin, you'll now have the same powerful threat detection capabilities for your Azure AKS clusters.

What is Falco?

Falco is a Cloud Native Computing Foundation project that provides runtime threat detection. Out of the box, Falco examines syscalls to alert you to any suspicious activity. And, since containers share the same kernel as their host, Falco can monitor not only activity on the host but also activity on all of the containers running on that host. Moreover, Falco pulls data from both Kubernetes and the container runtime to add additional context to its alerts.

With Falco running on your GKE clusters you can be notified of a wide variety of events, such as:

Did someone start a container with high privileges?
Has someone shelled into a running container?
Has an executable been added to the container after it was deployed?

These are just a few examples. Falco has over 80 rules that can be used to make you aware of not only external threats but also when clusters aren't being operated in accordance with industry best practices.

What is the AKS audit logs plugin?

The AKS audit logs plugin extends Falco's capabilities to Microsoft Azure Kubernetes Service (AKS) clusters, providing you with the same security insights and threat detection Falco already offers for GKE and EKS environments. With this plugin, you can seamlessly integrate AKS audit logs into Falco's event processing pipeline, enabling it to identify anomalies, suspicious activities, and policy violations within your AKS-based workloads.

Using AKS audit logs plugin

In order to use the AKS audit log plugin, you must first configure your AKS cluster to ship the logs where we can fetch them.

The current supported output source is Event hub, so when following the guide to configure your AKS audit logs, you must have Eventhub enabled. You can also optionally send it to other sources:

Once you have the stream enabled, you must create or reuse a storage account blob container so that the plugin can track the last event that was consumed, which is done trough checkpoints.

Configuring Falco to use AKS audit logs plugin

First, using falcoctl, download the plugin:

sudo falcoctl artifact install k8saudit-aks

In your falco.yaml file, you must add the plugin configuration and later enable the plugin

config_files:
 - /etc/falco/config.d

watch_config_files: true

plugins:
# - name: k8saudit
# library_path: libk8saudit.so
# init_config: ""
# open_params: "http://:9765/k8s-audit"
# - name: json
# library_path: libjson.so
 - name: k8saudit-aks
 library_path: libk8saudit-aks.so
 init_config:
 event_hub_name: ${EVENTHUB_NAME}
 blob_storage_container_name: ${BLOB_STORAGE_CONTAINER_NAME}
 event_hub_namespace_connection_string: ${EVENTHUB_NAMESPACE_CONNECTION_STRING}
 blob_storage_connection_string: ${BLOB_STORAGE_CONNECTION_STRING}

load_plugins: [k8saudit-aks]

stdout_output:
 enabled: true

Once they are exported, run Falco and after some seconds you'll logs informing the k8saudit-aks plugin was loaded:

falco -c /etc/falco/falco.yaml -r /etc/falco/falco_rules.yaml

Tue Dec 17 18:02:07 2024: Opening 'k8s_audit' source with plugin 'k8saudit-aks'
2024/12/17 21:02:07 [k8saudit-aks] opened connection to blob storage
2024/12/17 21:02:07 [k8saudit-aks] opened blob checkpoint connection
2024/12/17 21:02:07 [k8saudit-aks] opened consumer client
2024/12/17 21:02:07 [k8saudit-aks] created eventhub processor

Testing out!

Append rule to falco_rules.yaml file:

- rule: K8s Audit Event Detected
 desc: A test rule that detects any Kubernetes audit event
 condition: ka.req exists
 output: "K8s Audit Event Detected: %ka.req"
 priority: DEBUG
 source: k8s_audit
 tags: [testing, k8s_audit]

$ falco -c /etc/falco/falco.yaml -r /etc/falco/falco_rules.yaml

Then, you should see initialization message, followed by some events from your AKS cluster. Since we have debug enabled, you should see some events from the aksService:

Thu Dec 19 11:44:55 2024: Falco version: 0.39.2 (aarch64)
Thu Dec 19 11:44:55 2024: Falco initialized with configuration files:
Thu Dec 19 11:44:55 2024: /etc/falco/config.d/engine-kind-falcoctl.yaml | schema validation: ok
Thu Dec 19 11:44:55 2024: /etc/falco/falco.yaml | schema validation: ok
Thu Dec 19 11:44:55 2024: System info: Linux version 6.8.0-51-generic (buildd@bos03-arm64-031) (aarch64-linux-gnu-gcc-13 (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #52-Ubuntu SMP PREEMPT_DYNAMIC Thu Dec 5 13:32:09 UTC 2024
Thu Dec 19 11:44:55 2024: Loading plugin 'k8saudit-aks' from file /usr/share/falco/plugins/libk8saudit-aks.so
Thu Dec 19 11:44:55 2024: Loading plugin 'json' from file /usr/share/falco/plugins/libjson.so
Thu Dec 19 11:44:55 2024: Loading rules from:
Thu Dec 19 11:44:55 2024: /etc/falco/falco_rules.yaml | schema validation: ok
Thu Dec 19 11:44:55 2024: /etc/falco/falco_rules.local.yaml | schema validation: none
Thu Dec 19 11:44:55 2024: /etc/falco/falco_aks_audit.yaml | schema validation: ok
Thu Dec 19 11:44:55 2024: The chosen syscall buffer dimension is: 8388608 bytes (8 MBs)
Thu Dec 19 11:44:55 2024: Starting health webserver with threadiness 4, listening on 0.0.0.0:8765
Thu Dec 19 11:44:55 2024: Loaded event sources: syscall, k8s_audit
Thu Dec 19 11:44:55 2024: Enabled event sources: k8s_audit, syscall
Thu Dec 19 11:44:55 2024: Opening 'k8s_audit' source with plugin 'k8saudit-aks'
2024/12/19 14:44:55 [k8saudit-aks] opened connection to blob storage
2024/12/19 14:44:55 [k8saudit-aks] opened blob checkpoint connection
2024/12/19 14:44:55 [k8saudit-aks] opened consumer client
2024/12/19 14:44:55 [k8saudit-aks] created eventhub processor
Thu Dec 19 11:44:55 2024: Opening 'syscall' source with modern BPF probe.
Thu Dec 19 11:44:55 2024: One ring buffer every '2' CPUs.

10:52:03.348668000: Debug K8s Audit Event Detected: verb=create, user=aksService, groups=(system:masters,system:authenticated), target=<NA>

Let's meet!

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Enjoy 😎,

Igor

Blog: How to Deploy Falco with k8s-metacollector + k8smeta Plugin

Mon, 14 Oct 2024 00:00:00 +0000

In today's cloud-native world, securing Kubernetes environments has become increasingly critical as containerized workloads gain complexity. Falco is designed to monitor and detect anomalous activities in Kubernetes clusters and container environments. By continuously observing system calls and enriching event data with metadata, Falco ensures that any suspicious behavior is detected in real-time, protecting against threats like privilege escalations, file tampering, and network anomalies.

In this tutorial, we will guide you through deploying Falco with two powerful components: k8s-metacollector and the k8smeta plugin. These tools significantly enhance Falco’s security event detection by adding important Kubernetes context, such as pod names, namespaces, deployment details, to the alerts. Additionally, we will explore how to leverage the new append_output feature introduced in Falco version 0.39.0. This feature allows you to append extra metadata fields to Falco’s output, without the need to modify your rules.

By the end of this guide, you will have a Falco setup capable of detecting security issues in Kubernetes with enriched metadata output, ensuring you get a complete picture of your cluster’s security posture. Whether you're an experienced Kubernetes administrator or just starting to explore container security, this guide will help you make the most of Falco's capabilities in a Kubernetes environment.

What You'll Learn:

The purpose and benefits of using the k8s-metacollector and k8smeta plugin to enrich Falco alerts with Kubernetes-specific data.
How to deploy Falco with the k8smeta plugin on a Kubernetes cluster.
How to configure and use the append_output feature to enhance Falco alerts with additional metadata fields.

Prerequisites:

A working Kubernetes cluster and some familiarity with Kubernetes concepts.
Basic knowledge of Falco and how it works.
Helm installed on your system (for easy deployment of Falco).

Let’s dive in and set up a Falco deployment that will give you deeper security insights for your Kubernetes workloads.

Step 1: Understanding k8s-metacollector and k8smeta Plugin

As Kubernetes has become the de facto platform for orchestrating containerized applications, it’s important to gain full visibility into what's happening within your cluster, especially when it comes to security monitoring. Falco can detect suspicious activities based on system calls, but to make these alerts more actionable, additional context about your Kubernetes resources (such as pod names, namespaces, and labels) is invaluable.

That’s where the k8s-metacollectorand k8smeta plugin come in.

What is the k8s-metacollector?

The k8s-metacollector is responsible for gathering Kubernetes metadata for security events and sending that information to Falco. It collects key information for different resources from your Kubernetes cluster, such as:

Pods;
Namespaces;
ReplicaSets;
Services;
Deployments;

The collected metadata provides greater clarity about where and why certain events are happening, which is crucial for pinpointing and mitigating security incidents in large-scale Kubernetes environments. Without this context, security alerts may lack the detail needed for quick and effective response.

What is the k8smeta Plugin?

The k8smeta plugin is a source plugin for Falco that works in tandem with the k8s-metacollector. While Falco generates alerts based on detected anomalies, the k8smeta plugin enriches these alerts with Kubernetes-specific metadata, which allows you to understand exactly which Kubernetes entities (pods, deployments, namespaces) are involved in the detected event. This context is vital when you're trying to correlate security incidents with the resources they affect.

Key benefits of the k8smeta plugin include:

Enriched Alerts: Falco alerts become more informative with Kubernetes-specific data like pod names, namespaces, and deployment names.
Improved Debugging: Knowing exactly which pod or namespace is involved in an alert can significantly reduce the time spent debugging and fixing security issues.
Event Correlation: The plugin makes it easier to correlate low-level system events with higher-level Kubernetes concepts, providing a clearer view of what's happening in your cluster.

By using the k8s-metacollector and k8smeta plugin together, you transform Falco’s raw system call data into rich, actionable insights that give you full visibility into your Kubernetes environment.

Step 2: Installing Falco, k8s-metacollector, and k8smeta Plugin with Helm and Configuring append_output

Deploying Falco along with the k8s-metacollector and the k8smeta plugin using Helm is a seamless process. This step will guide you through adding the Falco Security Helm chart repository, installing Falco, enabling the k8s-metacollector, and configuring the append_output feature to append Kubernetes metadata to Falco alerts.

Step 2.1: Add the Falco Helm Chart Repository

Before you install Falco, you need to add the official Falco Security Helm chart repository to your Helm setup. Run the following command:

helm repo add falcosecurity https://falcosecurity.github.io/charts

Update your local Helm repositories to ensure you’re using the latest chart version:

helm repo update

Step 2.2: Install Falco with k8s-metacollector and append_output

With the repository added, use the following command which includes the additional settings to enable the collection of Kubernetes metadata and to append this metadata to Falco alerts:

helm install falco falcosecurity/falco \
 --version 4.11.1 \
 --namespace falco \
 --create-namespace \
 --set collectors.kubernetes.enabled=true \
 --set tty=true \
 --set-json 'falco.append_output=[{"match": {"source": "syscall"},"extra_output": "pod_uid=%k8smeta.pod.uid, pod_name=%k8smeta.pod.name, namespace_name=%k8smeta.ns.name"}]'

Breaking Down the Command:

helm install falco falcosercurity/falco: Installs Falco using the latest chart from the Falco Security repository.
--version 4.11.1: Uses the 4.11.1 version of the chart. At the writing time it's the latest version.
--namespace falco: Deploys Falco into the falco namespace. This helps keep Falco’s resources organized separately from other applications.
--create-namespace: Automatically creates the falco namespace if it doesn’t already exist.
--set collectors.kubernetes.enabled=true: Enables the k8s-metacollector and configures the k8smeta plugin.
--set tty=true: Ensures that Falco logs are emitted as soon as possible.
--set-json 'falco.append_output=...': Configures the append_output feature to append specific Kubernetes metadata fields to Falco’s alerts.

Why Use the append_output Feature?

The append_output feature allows you to enrich Falco alerts with additional metadata, providing a clearer view of which Kubernetes resources are involved in each security event. This context helps security teams quickly understand the severity and scope of an incident.

For example, an alert will now include:

pod_uid: To precisely identify the pod.
pod_name: To know which pod triggered the alert.
namespace_name: Namespace where the pod is running.

Step 2.3: Verifying the Installation

Once the installation is complete, you can verify that Falco and the k8s-metacollector are working as expected by checking the status of the Falco pod in the Falco namespace:

kubectl get pods -n falco

You should see the Falco pods running successfully.

Step 3: Testing the Setup

Now that everything is in place, it's time to test the setup by deploying a simple Nginx pod and triggering Falco to generate security alerts enriched with Kubernetes metadata.

Step 3.1: Deploy an Nginx Pod

To create some activity that Falco can monitor, start by deploying an Nginx pod in the falco namespace:

kubectl run nginx --image=nginx --namespace falco

This command will launch an Nginx container in the falco namespace.

Step 3.2: Wait for the Nginx Pod to Run

Confirm that the Nginx pod is up and running by checking its status:

kubectl get pods -n falco

Once the pod is in the Running state, you can proceed to the next step.

Step 3.3: Exec Into the Nginx Pod to Trigger Alerts

Exec into the running Nginx pod to simulate an interactive terminal session, which is something Falco is configured to detect:

kubectl exec -it nginx -n falco -- /bin/bash

This command opens a shell session inside the Nginx container. Inside the container, run some basic commands like ls or echo to generate system calls that Falco can monitor.

Step 3.4: Check Falco Logs for Alerts

After executing inside the Nginx pod, check the Falco logs to see if any alerts were triggered by the kubectl exec action:

kubectl logs -n falco -l app.kubernetes.io/name=falco

In the logs, you should see alerts related to the interactive terminal session such as:

13:18:57.434030270: Notice A shell was spawned in a container with an attached terminal (evt_type=execve user=root user_uid=0 user_loginuid=-1 process=bash proc_exepath=/usr/bin/bash parent=containerd-shim command=bash terminal=34816 exe_flags=EXE_WRITABLE|EXE_LOWER_LAYER container_id=7cff9da475c6 container_image=docker.io/library/nginx container_image_tag=latest container_name=nginx k8s_ns=falco k8s_pod_name=nginx) pod_uid=2f20370c-6e0b-44b8-8ea1-2aa786d80f13, pod_name=nginx, namespace_name=falco

This confirms that Falco is properly configured to detect activity inside the pod and append useful Kubernetes metadata to the alerts.

Key Takeaways:

In this tutorial, we explored how to deploy Falco with the k8s-metacollector and k8smeta plugin to enhance security monitoring in a Kubernetes environment. By enabling Falco’s append_output feature, we were able to enrich security alerts with vital Kubernetes metadata such as pod UID, pod name, and namespace, making the alerts more actionable and informative.

Enhanced Alert Context: By appending Kubernetes metadata, you get more contextualized and meaningful alerts, enabling better incident investigation and faster resolution.
Seamless Integration: Thanks to Helm, deploying Falco alongside the k8s-metacollector and k8smeta plugin is easy and efficient, requiring just a few simple commands.
Real-Time Threat Detection: Falco continuously monitors system calls and Kubernetes events in real-time, ensuring that you’re always aware of potentially suspicious or malicious activities within your cluster.

Blog: Falco at the KubeCon NA 2022

Tue, 08 Nov 2022 00:00:00 +0000

It was KubeCon recently. I doubt anyone reading this didn't know about it. And if you attended, you're probably still receiving e-mails about the event.

KubeCon is where everyone wants to be. And Falco was there too. It did indeed have a great presence: A project meeting, mentions, a few presentations, a keynote, it even had a party!

Once there, it was Falco time!

Project Meeting

A project meeting is where maintainers of the project, users, adopters and contributors have the opportunity to exchange impressions. On Tuesday afternoon, Falco maintainers met with interested users and potential adopters, and presented, not only the background of the project, but also its future roadmap.

There were questions from the attendees, requests and announcements of upcoming features, and even live demos. From the new plugins framework till the recent gVisor support, including a deep explanation of Falco libraries' insights. If you didn't know how Falco worked internally, you could leave the room being an expert.

Presentations

Falco is a well known project. It was mentioned in at least five presentations. Some of these, delivered by the core maintainers. Others, by the community or the CNCF organization itself. Its presence in so many occasions reflected the project's reputation in the community.

Keynote

Tuesday morning. Still tired from the jet-lag, and after the first day of Cloud Native SecurityCon, our first public Falco moment of the day: A Keynote at the SecurityCon delivered by Loris Degioanni, original creator of Falco.

Loris introduced the new GitHub Plugin for Falco, which is capable of detecting events like using GitHub actions for cryptominers, pushing code with secrets, or even detecting when someone starred the repository.

The time dedicated to a keynote is usually short, but for Loris it seemed to be enough to perform a couple of live demos. Don't miss them in this video.

Detecting Threats in GitHub with Falco - Loris Degioanni

The Eye of Falco

That same day, a few hours later, Stefano Chierici, Senior Security Researcher, and Lorenzo Susini, Open Source Engineer, both contributors of Falco, presented one of the most exciting of its features: Detection of attempts to escape Linux capabilities.

During this presentation, Lorenzo did an extensive walkthough on Linux capabilities, explaining the security situation before having them, detailing on its different sets (effective, permitted and inheritable) and its security implications when creating new processes that require higher privileges.

Stefano, on the other side, walked us through different scenarios showing a variety of real attacks. Therefore, having the CAP_SYS_MODULE capability enabled in the container would allow an attacker to use a Kernel Module to attack; having the CAP_SYS_PTRACE capability active would allow the injection of malicious code into memory; and having the CAP_SYS_ADMIN capability might open more than one path to make our host exploitable.

Trying not to spoil the end of the presentation (you can already imagine it though), we recommend to watch the following video to see how Falco faces this kind of threats, as it does with many others, by obtaining the state of the container and warning the user if the capabilities exceed the desirable ones.

The Eye of Falco: You Can Escape but Not Hide - Stefano Chierici & Lorenzo Susini

Detecting the Undetectable

Falso also squeezed into a presentation from Carol Valencia, Cloud Native Security Advocate at Aqua Security, where she demonstrated how three different runtime security solutions, Falco among them, were able to detect fileless attacks.

Fileless Attack - Detecting the Undetectable

Falco Project Updates

For those that were not able to attend the Project Meetings at the SecurityCon, KubeCon was a second great opportunity to learn from their favority CNCF projects.

On Friday afternoon, Jason Dellaluce and Luca Guerra, both Open Source Engineers, as well as Falco maintainers, gave an overview of the Falco project and its recent updates.

Security In the Cloud With Falco: Overview And Project Updates - Jason Dellaluce & Luca Guerra

Falco Kiosk at the CNCF Pavillion

This year at the KubeCon, Falco maintainers spent a good amount of time at the Falco kiosk. They received visitors interested in the project, some, already users of Falco, others, new users looking to learn about it, even a couple of youtubers asking to interview the maintainers for their channels.

All in all, an awesome chance to discover, first hand, what people really thought of Falco, wonders and pain points, good and not so good experiences with the tool and its ecosystem. In other words, real and valuable feedback.

Book signing

We haven't mentioned it yet, but Falco even had a book at the KubeCon. Shortly before the event, O'Reilly published Practical Cloud Native Security with Falco, written by Loris Degionni and Leonardo Grasso, both Falco maintainers with a large experience in the project.

Wednesday and Thursday, Loris and Leo spent some time signing copies of their book to users and developers interested in learning the secrets of Falco. Receiving a book at the KubeCon is probably not such a highligh anymore. Receiving Falco users willing to queue to receive your book is still a rewarding experience though.

Party

KubeCon is not only about collecting swag and attending presentations, although these are a great source of knowledge (and the swag a lot of stolen space in your luggage). KubeCon is also about interacting with other attendees, having exciting conversations, sharing experiences and point of views.

So Falco thought of using an evening to do exactly that!

People at the event (let's call it a party!). So people at the party enjoyed some food, drinks, had meaningful conversations -at least, that's what we want to believe-, and played Cards against Containers -the paper version, not the online one.

Since the party took place on Tuesday, it meant a nice break between two days full of Security-related presentations, and the KubeCon starting the next day. We didn't stay long, but we had some joy.

Conclusion

As you can see, it was a week full of emotions, opportunities, friends and colleagues, and Falco. We are already looking forward to the next event, and we hope you too.

And if you didn't get your copy of the book, maybe there'll be another chance next year in Seattle or Amsterdam ;-)