Falco – Release

Blog: Introducing Falco Operator 0.2.0

Mon, 23 Mar 2026 00:00:00 +0000

Dear Falco Community, today we are excited to announce the release of Falco Operator 0.2.0, the first production-ready release of the Kubernetes operator for Falco!

Since the technical preview announced with Falco 0.41.0, we have been working hard to make the operator robust, extensible, and ready for real-world environments. This release brings a redesigned API, a new Component controller for managing the Falco ecosystem, new artifact management capabilities, enhanced observability, and a significantly improved operational model, all grounded in Kubernetes-native patterns.

We merged 58 commits since v0.1.1, delivering major new features, 10 bug fixes, and comprehensive architectural improvements. Thank you to all our contributors and the community for your feedback along the way!

Going forward, the Falco Operator is the recommended way to deploy and manage Falco on Kubernetes. While the existing Helm chart remains fully supported, we plan to transition to the operator as the standard deployment method. More details on the transition timeline will follow in a future announcement.

To learn everything about the changes, read on!

What's new? TL;DR

Key features:

Ecosystem components - deploy Falcosidekick, Falcosidekick UI, and k8s-metacollector as managed components
ConfigMap support for rules and configuration, alongside OCI artifacts and inline definitions
Structured API types for inline rules and configuration - YAML objects instead of strings
Redesigned OCI artifact API with separate image and registry configuration
Reference tracking with finalizers to prevent accidental deletion of Secrets and ConfigMaps
Enhanced observability with Kubernetes events and status conditions across all controllers
Update strategy support for DaemonSet and Deployment modes
Server-Side Apply migration for safer, conflict-free reconciliation

Key fixes:

Plugin initConfig now supports nested configuration objects
RBAC compatibility with Kubernetes 1.32+
Spurious update prevention via managed fields comparison
Correct event recording with node-level attribution

This release comes with breaking changes that require updating your existing custom resources before upgrading. Please read the migration guide carefully before proceeding.

The road to production readiness

When we introduced the Falco Operator as a technical preview in Falco 0.41.0, the vision was clear: provide a Kubernetes-native way to deploy and manage Falco that goes beyond what Helm charts and static manifests can offer. Since then, every aspect of the operator has been refined.

The reconciliation logic now uses Server-Side Apply for conflict-free updates. Status conditions follow Kubernetes conventions (Programmed, ResolvedRefs, Available, Reconciled) so that standard tooling and dashboards can monitor operator health out of the box. Finalizers protect referenced resources from accidental deletion. And the new Component controller lays the groundwork for managing the entire Falco ecosystem from a single operator.

Version 0.2.0 is the result of this effort, a release we are confident in for production environments.

Major features and improvements

Ecosystem components

The new Component custom resource (instance.falcosecurity.dev/v1alpha1) enables the operator to deploy and manage the full Falco ecosystem from a single control plane. Three component types are supported:

Type	Component	What it does
`metacollector`	k8s-metacollector	Centralized Kubernetes metadata for Falco instances
`falcosidekick`	Falcosidekick	Fan-out daemon - routes Falco events to 70+ integrations (Slack, Elasticsearch, S3, Kafka, and more)
`falcosidekick-ui`	Falcosidekick UI	Web dashboard for real-time event visualization

Deploying Falcosidekick is as simple as:

apiVersion: instance.falcosecurity.dev/v1alpha1
kind: Component
metadata:
 name: sidekick
spec:
 component:
 type: falcosidekick
 replicas: 2

The operator handles the Deployment, Service, ServiceAccount, and RBAC automatically. Each component type ships with production-ready defaults (health probes, security context, resource profiles) that can be fully customized via podTemplateSpec.

For Falcosidekick UI, note that an external Redis instance is required. If Redis is not available, the pod stays in Init:0/1 state, the built-in wait-redis init container blocks until Redis is reachable. See the component documentation for setup examples including a bundled Redis StatefulSet.

As part of this work, the internal controller structure was reorganized under controllers/instance/ with shared reconciliation logic extracted into reusable packages, improving maintainability and consistency across all instance-level controllers.

ConfigMap support for rules and configuration

Rulesfile and Config resources can now source their content from Kubernetes ConfigMaps, in addition to OCI artifacts and inline definitions. This provides a familiar, Git-friendly workflow for teams that manage configuration through standard Kubernetes tooling.

Rulesfile from a ConfigMap:

apiVersion: artifact.falcosecurity.dev/v1alpha1
kind: Rulesfile
metadata:
 name: custom-rules
spec:
 configMapRef:
 name: my-rules-configmap
 priority: 50

Config from a ConfigMap:

apiVersion: artifact.falcosecurity.dev/v1alpha1
kind: Config
metadata:
 name: custom-config
spec:
 configMapRef:
 name: my-config-configmap
 priority: 50

The referenced ConfigMap must contain the content under a key named rules.yaml (for Rulesfile) or config.yaml (for Config). All three sources (OCI, inline, and ConfigMap) can be combined in a single resource when needed.

Structured API types

The inlineRules field in the Rulesfile CRD and the config field in the Config CRD are now structured YAML objects instead of plain strings. This enables proper validation, better editor support, and eliminates the need for YAML-in-YAML escaping.

Before (v0.1.x):

spec:
 config: |-
 engine:
 kind: modern_ebpf

After (v0.2.0):

spec:
 config:
 engine:
 kind: modern_ebpf

The same applies to inlineRules: rules are now defined as structured YAML lists rather than pipe-literal strings.

Redesigned OCI artifact API

The OCI artifact reference model has been completely redesigned for clarity and extensibility. The previous flat reference and pullSecret fields are replaced with a structured image and registry model.

Before (v0.1.x):

spec:
 ociArtifact:
 reference: ghcr.io/falcosecurity/rules/falco-rules:latest
 pullSecret:
 secretName: my-secret

After (v0.2.0):

spec:
 ociArtifact:
 image:
 repository: falcosecurity/rules/falco-rules
 tag: latest
 registry:
 name: ghcr.io
 auth:
 secretRef:
 name: my-secret

This separation makes the API more explicit and enables per-registry TLS configuration, plain HTTP support, and a consistent credential model. See the migration guide for details on updating your resources.

Reference tracking with finalizers

The operator now adds finalizers to Secrets and ConfigMaps that are referenced by artifact resources. This prevents accidental deletion of credentials or configuration data that would break Falco deployments. When a referenced resource is deleted, the operator blocks the deletion until all referencing artifacts are updated or removed.

Enhanced observability

All controllers now emit Kubernetes events for significant operations: artifact creation, updates, removals, and priority changes. Events include the node name for artifact controllers, making it straightforward to trace which operations happened on which nodes.

Status conditions have been overhauled to follow Kubernetes conventions:

Artifact resources report Programmed (whether the artifact is successfully applied) and ResolvedRefs (whether referenced ConfigMaps/Secrets exist)
Falco instances report Reconciled and Available
All artifact CRDs now include printcolumns for readable kubectl get output

Update strategy support

The Falco CRD now accepts update strategy configuration for both deployment modes:

# DaemonSet mode
spec:
 type: DaemonSet
 updateStrategy:
 type: RollingUpdate
 rollingUpdate:
 maxUnavailable: 1

# Deployment mode
spec:
 type: Deployment
 strategy:
 type: RollingUpdate
 rollingUpdate:
 maxUnavailable: 1
 maxSurge: 1

Server-Side Apply

Under the hood, the operator has migrated from the dry-run/update pattern to Server-Side Apply (SSA) for all reconciliation operations. This brings:

Conflict detection: concurrent modifications to managed fields are detected and reported
Ownership tracking: the operator only manages fields it owns, leaving user-applied changes intact
Reduced spurious updates: managed fields comparison prevents unnecessary API calls

Breaking changes ⚠️

Version 0.2.0 includes several API breaking changes. All existing custom resources must be updated before upgrading. A detailed migration guide is available in the repository documentation.

Summary of breaking changes

Change	Impact	Migration
`ociArtifact.reference` replaced by `ociArtifact.image` + `ociArtifact.registry`	All Rulesfile and Plugin CRs using OCI artifacts	Split the reference into `image.repository`, `image.tag`, and `registry.name`
`ociArtifact.pullSecret` replaced by `ociArtifact.registry.auth.secretRef`	CRs with private registry credentials	Update the credential reference path
Config `spec.config` changed from string to structured YAML	All Config CRs	Remove the `\|-` pipe literal, write YAML directly
Rulesfile `spec.inlineRules` changed from string to structured YAML	Rulesfile CRs with inline rules	Remove the `\|-` pipe literal, write YAML directly
Plugin `spec.config.initConfig` changed from `map[string]string` to JSON	Plugin CRs with init config	Re-apply CRD; flat maps still validate
Falco resource `shortName` changed from `prom` to `falco`	Scripts using `kubectl get prom`	Use `kubectl get falco` instead
Condition types renamed (`ConditionReconciled` → `Reconciled`, `ConditionAvailable` → `Available`)	Monitoring tools filtering on condition types	Update condition type filters
`kubectl get` column output changed for all CRDs	Dashboard parsing or scripts	Update parsers to match new column names
RBAC permissions expanded	Security-sensitive environments	Review the updated ClusterRole

After upgrading, re-apply all CRDs and update your custom resources following the migration guide. The operator will reconcile the new format automatically.

A Helm chart is on its way

We are currently developing a Helm chart for installing the Falco Operator itself, which will simplify deployment and configuration of the operator in production environments. Stay tuned for the announcement.

Meet us at KubeCon

We will be talking about the Falco Operator during the maintainer track at the upcoming KubeCon. If you are interested in learning more, asking questions, or sharing feedback, come find us at the CNCF Falco kiosk, we would love to chat!

Try it out

Install the operator:

VERSION=latest
if [ "$VERSION" = "latest" ]; then
 kubectl apply --server-side -f https://github.com/falcosecurity/falco-operator/releases/latest/download/install.yaml
else
 kubectl apply --server-side -f https://github.com/falcosecurity/falco-operator/releases/download/${VERSION}/install.yaml
fi

Then choose how you want to get started:

Full stack quickstart

Deploy the entire Falco ecosystem in the falco namespace with one command:

VERSION=latest
if [ "$VERSION" = "latest" ]; then
 kubectl apply --server-side -f https://github.com/falcosecurity/falco-operator/releases/latest/download/quickstart.yaml
else
 kubectl apply --server-side -f https://github.com/falcosecurity/falco-operator/releases/download/${VERSION}/quickstart.yaml
fi

This deploys Falco, container and k8smeta plugins, detection rules, Falcosidekick, Falcosidekick UI with Redis, and k8s-metacollector - all pre-wired.

Verify:

kubectl get falco,plugins,rulesfiles,configs,components -n falco
kubectl get pods -n falco

Step by step

Deploy Falco:

cat <<EOF | kubectl apply -f -
apiVersion: instance.falcosecurity.dev/v1alpha1
kind: Falco
metadata:
 name: falco
spec: {}
EOF

Add the container plugin (required by the official rules for container metadata fields):

cat <<EOF | kubectl apply -f -
apiVersion: artifact.falcosecurity.dev/v1alpha1
kind: Plugin
metadata:
 name: container
spec:
 ociArtifact:
 image:
 repository: falcosecurity/plugins/plugin/container
 tag: latest
 registry:
 name: ghcr.io
EOF

And add detection rules:

cat <<EOF | kubectl apply -f -
apiVersion: artifact.falcosecurity.dev/v1alpha1
kind: Rulesfile
metadata:
 name: falco-rules
spec:
 ociArtifact:
 image:
 repository: falcosecurity/rules/falco-rules
 tag: latest
 registry:
 name: ghcr.io
 priority: 50
EOF

Optionally, add Falcosidekick to route events to your favorite integrations:

cat <<EOF | kubectl apply -f -
apiVersion: instance.falcosecurity.dev/v1alpha1
kind: Component
metadata:
 name: sidekick
spec:
 component:
 type: falcosidekick
 replicas: 2
EOF

For the complete documentation, including the CRD reference, configuration options, and architecture overview, visit the Falco Operator repository and the operator setup guide.

Stay connected

Join us on social media and in our community calls! It's always great to have new members in the community, and we're looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco 0.43.0

Mon, 26 Jan 2026 00:00:00 +0000

Dear Falco Community, we are happy to announce the release of Falco 0.43.0 today!

This is a stabilization release that consolidates the changes introduced in 0.42.0, including the drop-enter initiative and the capture recording feature. It also introduces several deprecations to improve maintainability and fixes minor issues across falcoctl, plugins, and libs.

During this release cycle, we merged:

31 PRs on Falco, including 11 release note-worthy changes
48 PRs on Falco libs, including 17 release note-worthy changes
8 PRs on Falco drivers, including 3 release note-worthy changes

We upgraded libs to version 0.23.1 and drivers to 9.1.0+driver. Thank you to our maintainers and contributors. This would not have been possible without your support and dedication!

To learn everything about the changes, read on!

What's new? TL;DR

Key fixes:

evt.arg.filename field reintroduction
Falcoctl signature verification fixes
overflow and NULL pointer dereferences fixes for the container plugin, shipped with plugins/container/0.6.1
race condition fix for the k8smeta plugin, shipped with plugins/k8smeta/0.4.1

This release also comes with breaking changes that you should be aware of before upgrading.

Latest updates

Deprecations

In Falco 0.43.0, we are announcing the deprecation of three significant components to streamline the project, reduce maintenance burden, and focus on modern, more efficient alternatives. All these components are stable, and considering that the deprecation is first enforced in this version, they could be removed at any future version starting from 0.44.0.

Legacy eBPF probe deprecation

The "legacy" eBPF probe (configured via engine.kind=ebpf) was the original eBPF implementation in Falco. It required compiling a specific probe for each kernel version, often necessitating the dynamic usage of the falco-driver-loader or pre-built drivers. The Modern eBPF probe (engine.kind=modern_ebpf), which leverages CO-RE (Compile Once – Run Everywhere), has reached maturity and feature parity. It offers superior stability, portability (no need to compile drivers on the fly), flexibility and performance. Maintaining two eBPF drivers splits engineering effort and complicates the codebase. Users currently using the legacy eBPF probe are strongly encouraged to switch to the Modern eBPF probe by setting engine.kind=modern_ebpf in their falco.yaml, or to engine.kind=kmod if the used kernel doesn't provide support for the modern eBPF probe.

See the relevant section of the deprecation proposal for the detailed motivation behind the deprecation.

gVisor deprecation

The gVisor engine is a dedicated, internal C++ implementation designed to monitor system calls from gVisor sandboxes leveraging events coming from gVisor itself through gRPC. There is evidence that this engine is little used. Moreover, gVisor doesn't provide all information required to build all supported event types, indeed resulting in a system call source not completely equivalent to the ones provided by drivers. Finally, it requires libs being dependent on protobuf, this latter introducing a non-negligible build time overhead and maintainability burden.

See the relevant section of the deprecation proposal for the detailed motivation behind the deprecation.

gRPC output and server deprecation

The gRPC output was implemented to allow external consumers to subscribe to a stream of Falco security alerts over a gRPC connection. It was notably utilized by tools like the event-generator (in test mode) and custom integrations requiring a streaming API for alerts. The gRPC output and the gRPC server embedded in Falco add substantial complexity to the core codebase, including dependencies on specific protobuf and gRPC framework versions in Falco and libs. Over time, it has become clear that the community prefers standard, widespread integration patterns for alert consumption - primarily HTTP and the ecosystem enabled by Falcosidekick. Users consuming alerts via gRPC should migrate to the HTTP output or use Falcosidekick to forward events to their destination of choice.

See the relevant section of the deprecation proposal for the detailed motivation behind the deprecation.

GPG key rotation

In anticipation of the previous GPG key's expiration in January 2026, we have rotated the GPG key used to sign the official RPM and DEB packages. Pre-existing Falco installations (installed via apt or yum before the rotation) must manually import the new GPG key. Failure to do so may result in errors during package updates or verification failures. Please follow the "Trust the falcosecurity GPG key" step in the official documentation for your package manager:

apt (Debian/Ubuntu): Install with apt
yum/dnf (CentOS/RHEL/Fedora): Install with yum

Notice that new installations following the current documentation will automatically receive the updated key bundle and do not require additional steps.

For more details see [TRACKING] [deadline 2026-01-17] Rotate public GPG key for RPM/DEB package signing.

Container plugin improvements

The container plugin, which extracts metadata from container runtimes to enrich Falco events, includes important updates in version 0.6.1 to enhance its API capabilities and performance. This release exposes container.id, container.image, container.name, and container.type through the table API and adds comprehensive logging across all engines, while also preventing allocations by extensively using zero-allocation tools offered by the C++ (like std::string_view) and avoiding reflex matcher allocations during resolve operations.

Falcoctl tweaks and improvements

`follow` polling interval increase to 1 week

About three years ago, we started distributing Falco artifacts (rules files and plugins) via ghcr.io, and later added automatic rule updates in falcoctl with a 6h check interval. With years of data now, it’s clear we don’t need checks that frequent: our rule updates happen far less often. Moreover, due to the growth of Falco adoption, these frequent checks are now hitting ghcr.io rate limit. These two reasons drove the decision to increase the artifact follow interval from 6h to 1 week.

For more details see chore(scripts/falcoctl): increase follow interval to 1 week and Falco's Helm chart changelog.

Dependency resolution improvements

The artifact installation logic has been reworked to handle dependencies and references correctly. Previously, dependencies could be duplicated or incorrectly resolved, and signature verification was skipped for full registry references. Now dependencies are properly deduplicated, all refs are correctly resolved, and signatures are verified for all resolved dependencies, not just the top-level artifacts. This provides end-to-end verification of the entire dependency chain.

For more details see Inefficient deduplication logic and incorrect input handling for dependency resolution

Support for cosign v3

Falcoctl now supports Cosign v3 bundle format for signature verification. This is the new standard for signing OCI artifacts, replacing the legacy .sig tag format.

What this means for you:

Artifacts signed with cosign v3 are now fully supported
Backward compatibility with cosign v2 signatures is maintained

For more details see feat: Upgrade to Cosign v3 with Bundle Format

Key fixes

`evt.arg.filename` field reintroduction

As part of the recent "drop enter" optimization initiative (which removed enter events for most syscalls to improve performance), the filename argument - historically available only in the enter event for execve and execveat - was inadvertently made unavailable. This caused a regression where specific context, such as the exact path provided to the syscall (e.g., a symlink path versus the resolved binary path), was lost in the remaining exit event.

In Falco 0.43.0 (via libs 0.23.0), this has been fixed. The filename argument is now correctly populated in the exit events for these syscalls. Users can once again access this data using the evt.arg.filename field in their rules, ensuring that the critical execution context is preserved without needing the deprecated enter events.

For more details see Missing "filename" argument to execve syscall in libscap 0.22.x.

Falcoctl signature verification fixes

Signature verification fix for full reference artifacts

Fixed an issue where signature verification was skipped for artifacts specified with a full registry reference ( e.g., ghcr.io/falcosecurity/plugins/plugin/container:0.4.1). Now all artifacts are verified regardless of how they are referenced.

Signature verification fix for authenticated registries

Signature verification now works correctly on private/authenticated registries. Previously, verification would fail with authentication errors even though the artifact pull succeeded, and credentials were not being passed to the signature verification component.

Supported authentication methods:

Basic auth (Docker credentials)
OAuth2 client credentials
GCP Workload Identity (for GKE deployments)

For more details see fix(signature): pass registry credentials to cosign for signature verification

Breaking changes and deprecations

This version includes breaking changes you should be aware of before upgrading.

Bump drivers minimum required kernel version to `3.10`

Falco 0.43.0 introduces a breaking change regarding the Falco drivers. Starting with drivers version 9.1.0+driver, the minimum required Linux kernel version has been bumped to 3.10. In practice, this only affects the kmod driver and means that the kernel module will explicitly fail to compile on kernels older than 3.10. This choice is motivated by the fact that even Linux 3.10 is a 12-year-old kernel, and its support ended in 2017: maintaining support for older kernels is a maintenance burden and limits progress. This change enables the team to focus on modernizing the codebase and improving stability for current environments.

Deprecation warnings

Falco 0.43.0 introduces several deprecation warnings to help users migrate to newer components:

Legacy eBPF probe deprecation: using the legacy eBPF probe (engine.kind=ebpf) will now generate warnings
gVisor engine deprecation: using the gVisor engine (engine.kind=gvisor) will now generate warnings
gRPC deprecation: using the gRPC output or the gRPC server (grpc_output.enabled=true or grpc.enabled=true), will now generate warnings

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

Stay connected

Join us on social media and in our community calls, held every other Wednesday! It's always great to have new members in the community, and we're looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco 0.42.0

Wed, 22 Oct 2025 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.42.0!

This release brings exciting new capabilities, including the capture feature, significant performance improvements, and important bug fixes that enhance Falco's capabilities. During this release cycle, we merged:

52 PRs on Falco, including 23 release note-worthy changes
110 PRs on Falco libs, including 47 release note-worthy changes
102 PRs on Falco drivers, including 29 release note-worthy changes

We upgraded libs to version 0.22.1 and drivers to v9.0.0+driver. Thank you to our maintainers and contributors. This would not have been possible without your support and dedication!

To learn everything about the changes, read on!

What's new? TL;DR

Key features:

Key fixes:

Fix thread table memory leak when parsing vfork (or equivalent clone/clone3 with CLONE_VFORK) exit from the caller process;
Enable handling of multiple actions configured with syscall_event_drops.actions;
Disable dry-run restarts when Falco runs with config-watching disabled;
Fix abseil-cpp for Alpine build;
Fix detection sandbox containers for CRI and containerd runtimes (container plugin);
Stability improvements for container plugin and static linking of libgcc/libstdc++ for legacy compatibility;

This release also comes with breaking changes that you should be aware of before upgrading.

Major features and improvements

The 0.42.0 release contains a new capture feature and significant performance improvements. Here is a list of the key new capabilities.

Capture recording feature

Falco 0.42.0 introduces the new capture recording feature, now available at sandbox maturity. This capability allows Falco to generate .scap files whenever a detection rule is triggered automatically.

Each capture contains a detailed trace of system calls around the event, providing forensic-level visibility into what happened. The recordings can be opened directly in Stratoshark for Wireshark-style analysis of runtime behavior.

The capture system is fully configurable: you can enable global recording or tie captures to specific Falco rules for targeted runtime snapshots.

When targeting specific Falco rules (by setting mode: rules, as shown in the configuration below), users can modify individual rules to enable capture by adding capture: true and optionally capture_duration to specific rules. For example:

 - rule: Suspicious File Access
 desc: Detect suspicious file access patterns
 condition: >
 open_read and fd.name startswith "/etc/"
 output: >
 Suspicious file access (user=%user.name command=%proc.cmdline file=%fd.name)
 priority: WARNING
 capture: true
 capture_duration: 10000 # Capture for 10 seconds when this rule triggers

This configuration will capture events for 10 seconds whenever the "Suspicious File Access" rule is triggered, overriding the default duration.

Find below the configuration snippet to enable the capture feature in falco.yaml:

capture:
 # -- Set to true to enable event capturing.
 enabled: false
 # -- Prefix for capture files. Falco appends a timestamp and event number to ensure unique filenames.
 path_prefix: /tmp/falco
 # -- Capture mode. Can be "rules" or "all_rules".
 mode: rules
 # -- Default capture duration in milliseconds if not specified in the rule.
 default_duration: 5000

Learn more at KubeCon + CloudNativeCon North America 2025:

Project Lightning Talk: When Falco Spots Trouble, The Shark Swims In - Gerald Combs, Falco Core Maintainer
Beyond the Cloud(s): Falco's Ascent in Performance and Deep Visibility - Leonardo Grasso & Leonardo Di Giovanna, Sysdig

Drop enter initiative

We've just shipped a significant performance improvement: syscall enter events have been completely removed from our event pipeline.

In Falco, each system call traditionally used to generate two events: an enter event when syscall kernel processing starts (i.e., before its arguments are processed) and an exit event when the kernel processing completes. Now that we collect all relevant information on exit events, we can drop the generation and processing of enter events.

Nevertheless, for TOCTOU (Time-of-Check to Time-of-Use) mitigation, a few selected enter events are still monitored internally — their relevant data is captured and stored — but these events are no longer pushed downstream to the userspace processing pipeline.

By focusing solely on syscall exit events, we've nearly halved the number of events generated and processed by userspace, eliminating redundant data collection. This reduces the Falco instrumentation overhead, improving workloads' performance up to 20% (by reducing syscall execution latency). It also decreases Falco's CPU usage up to 30%, especially in high-syscall environments.

From a developer's perspective, this also removes ambiguity about where syscall parameters should be defined, streamlines event processing logic, and makes event handling code cleaner and easier to maintain.

Overall, you can expect better performance, leaner code, and a more predictable event model moving forward.

For more details, see:

Plugin event schema versioning

Falco 0.42.0 introduces plugin event schema validation, enabling plugins to specify their compatible event schema version.

It provides an event schema validation system for syscall events consumed by plugins that offer parsing and/or field extraction capabilities, ensuring backward compatibility and clear error reporting for plugins that depend on specific Event Schema Versions.

If the plugin does not declare a required Schema Version, it is assumed to be compatible with 3.0.0, the initial major version when the plugin event schema validation was introduced.

The plugins should implement a new function of the Plugin API to declare the required schema version. Find below the signature of the new API function:

// New plugin API functions for schema management
typedef struct {
...
// Event schema version check
//
// Return the minimum event schema version required by this plugin.
// Required: no
// Arguments:
// - s: the plugin state, returned by init(). Can be NULL.
// Return value: the event schema version string, in the following format:
// "<major>.<minor>.<patch>", e.g. "4.0.0".
// If the function is not implemented or NULL is returned, the plugin is assumed to be
// compatible with schema version 3.0.0.
//
const char* (*get_required_event_schema_version)(ss_plugin_t* s);
} plugin_api;

For more details, see:

Plugin system event schema versioning proposal

Thread table auto-purging configuration

We've added a few new falco_libs configurations for advanced users who want finer control over Falco's performance and resource usage. It introduces tunable parameters for Falco's internal thread table, which tracks active threads:

thread_table_size defines the maximum number of entries.
thread_table_auto_purging_interval_s controls how often stale threads are cleaned up.
thread_table_auto_purging_thread_timeout_s sets how long inactive threads are kept before removal.

These options let you balance memory efficiency, CPU usage, and state accuracy, with related metrics (n_drops_full_threadtable, n_store_evts_drops) available to guide tuning.

Static fields

Falco 0.42.0 adds a new static_fields configuration object allowing users to add statically defined fields to the Falco engine. The following example illustrates how to specify new static fields:

static_fields:
 foo: bar
 foo2: ${bar2}

Notice that foo2: ${bar2} leverages the Falco's behavior of expanding env variables in config YAML.

After specifying them, these fields can be used in normal rule conditions, by prepending the static. prefix (e.g.: evt.type=open and static.foo=bar). Moreover, if append_output.suggested_output is true, they'll be automatically appended to each rule output, in the form static_foo=bar.

For more details, see:

Breaking changes and deprecations ⚠️

This version comes with breaking changes that you should be aware of before upgrading.

Event direction and `evt.dir` deprecation

Following the enter events initiative, the evt.dir field, as well as the concept of "direction", have been deprecated in Falco 0.42.0 and will be removed in a future release. Until field removal and since Falco 0.42.0, specifying evt.dir='>' will match nothing, while specifying evt.dir='<' will match everything, with a warning informing the user about the deprecation. Users are encouraged to get rid of any reference to evt.dir, as its presence will result in an error at rules loading time after its removal.

Plugin API changes

Old plugins consuming syscall events not declaring the required event schema version will be incompatible with Falco 0.42.0 and later.

Deprecation warnings

Falco 0.42.0 introduces several deprecation warnings to help users migrate to newer APIs:

evt.dir field deprecation: Rules using the deprecated evt.dir field will now generate warnings;
Enter events drop stats: Prometheus metrics for enter events drop statistics have been deprecated;
Configuration warnings: Enhanced warning system for deprecated configuration options;

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

Stay connected

Join us on social media and in our community calls, held every other Wednesday! It's always great to have new members in the community, and we're looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco 0.41.0

Thu, 29 May 2025 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.41.0!

This version brings several new features, performance enhancements, and bug fixes that streamline Falco’s detection capabilities. During this release cycle, we merged more than 50 PRs on Falco and around 130 PRs for libs and drivers, version 0.21.0 and version 8.1.0, respectively. Thank you to our maintainers and contributors. This would not have been possible without your support and dedication!

To learn everything about the changes, read on!

What’s new? TL;DR

Key features:

Reimplemented container engines support from scratch;
A Kubernetes operator is taking shape;
Falco's config_files configuration gained support to specify the merge strategy;
Modern eBPF driver is now capable of trying to load multiple programs for each event; consequently, sendmmsg and recvmmsg will now make use of bpf_loop eBPF helper where available, boosting their performances;
New proc.aargs field available, ie: a lookup for an ancestor args field;
proc.args gained support for indexed access, to only check a certain argument;
json_include_output_fields configuration key for Falco to control whether output fields are included in the JSON message;
Ongoing work to improve libs code modularity;

Key fixes:

Avoid kmod crashing when a CPU gets enabled at runtime;
Fixed Falco Prometheus metrics with multiple event sources enabled;
Fixed RPM packages evaluation of RPM scripts;
-o options do now correctly override included config_files;

This release also comes with breaking changes that you should be aware of before upgrading.

Major features and improvements

The 0.41.0 release contains a number of features and UX improvements. Here is a list of some of the key new capabilities.

Reimplemented container engines support

In the Falco 0.41.0 release, the Falco team has completely revised its support for container engines. Key improvements include:

Container support is now a plugin;
The plugin will attach a listener to the engine's SDKs onCreate signal; since onCreate comes way before onStart, we have plenty of time to deliver the container's metadata before the first process in the container is even started;
For now, it is bundled within Falco to avoid breaking changes, but in the future, it will need to be downloaded through falcoctl;

These changes should address all issues related to missing container metadata.

Kubernetes operator

In Falco 0.41.0, we worked hard to create a Falco k8s operator: https://github.com/falcosecurity/falco-operator/. For now, this is considered a technical preview, but we will deliver a fully functional operator very soon. Expect more news in a new blog post!

Security best practices improvements

We are grateful for the suggestions we received from security experts and adopters in our community, and so we implemented the following enhancements:

The modern eBPF probe will no longer store security sensitive settings in the .bss mmapable segment but will use dedicated maps instead. This is a security best practice because it prevents other processes running with elevated privileges from tampering with the map file descriptor, which would be harder to detect. We would like to thank Mouad Kondah for suggesting this change!

Falco will no longer consider rule files or contents of rule directories that do not have a .yml/.yaml extension. This prevents accidental processing of files that are not related to rules. We would like to thank our user Travis Smith for suggesting this change!

Breaking changes and deprecations ⚠️

This version comes with breaking changes, mostly in the configuration interface.

Removed command line options and equivalent configuration options

We removed the already deprecated options -S/--snaplen, -A, and -b, and it is now possible to achieve the same result through the Falco configuration:

for -S/--snaplen: falco_libs.snaplen config key;
for -A: base_syscalls.all config key;
for -b: buffer_format_base64 config key;

The configuration options for the container engines, added in 0.40.0, have been completely dropped in favor of the new plugin init configuration which can be found at https://github.com/falcosecurity/plugins/tree/main/plugins/container#configuration.

You can find more information on breaking changes in the tracking issue.

Behavior changes

Falco will now only consider and consequently load rules whose name ends in .yml or .yaml.

Dropped features

syslog related fields were dropped by libs, since they were unused.

Also, as a consequence of the new container plugin, some breaking changes had to take place:

the musl build is inherently not able to load plugins; that means that it loses container metadata support;
falcosecurity_scap_n_containers and falcosecurity_scap_n_missing_container_images metrics are now moved to the plugin, and their name now have the falcosecurity_plugins_ prefix;
-pc and -pk command line options are now ineffective; it is up to the container and k8smeta plugins to declare suggested fields to be used as output fields; consequently, container_image=%container.image.repository and k8s_ns=%k8s.ns.name changed their name to container_image_repository= and k8s_ns_name=;

Deprecations

In Falco 0.41.0, we have deprecated the following options:

-p cli flag; the only remaining user for it is gVisor, which will be ported to a plugin sooner or later and will then make use of the suggested output fields plugin API;

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What’s next?

Falco is more mature with each release. Following its graduation, we have published the roadmap for version 1.0.0, which is guiding us in the next steps. For the next release, you can expect more stability, a refined k8s operator, improved performance, and, as always, new detections and fixes.

Stay connected

Join us on social media and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Falco Talon v0.3.0

Tue, 11 Feb 2025 00:00:00 +0000

Today, we announce the release of Falco Talon 0.3.0 🦅!

Three updates in a row, after Falco and Falcosidekick, it's time for Falco Talon to know a new version.

What's new?

The key feature this release brings is the new actionner kubernetes:sysdig. For those who are not familiar with sysdig, it's a CLI tool that allows to capture and record the syscalls, like tcpdump does for the network packets. Old brother of Falco, they share the same libs and filters.

With this new integration, when a suspicious event occurs in a pod, Talon triggers a capture and then exports the created artifact to AWS S3 or Minio. You can configure the duration and the amount of bytes captured for each syscall. Check out the docs to discover more settings.

See this example rule:

- action: Capture the syscalls
 actionner: kubernetes:sysdig
 parameters:
 buffer_size: 2048
 duration: 20
 output:
 target: minio:s3
 parameters:
 bucket: falco-talon
 prefix: /sysdig/

After the action has been completed, you'll find the capture in Minio:

And you can run the CLI tool sysdig to explore it:

❯ sysdig -r 2025-01-23T13-26-41Z_default_cncf-597d69dbd4-h9fcb_sysdig.scap.gz evt.type=execve and evt.dir=">"

18563 14:26:38.376178286 0 bash (616444.616444) > execve filename=/usr/bin/apt
19163 14:26:38.394972623 0 apt (616445.616445) > execve filename=/usr/bin/dpkg
19599 14:26:38.399546432 0 apt (616446.616446) > execve filename=/usr/lib/apt/methods/http
20319 14:26:38.408846350 0 apt (616447.616447) > execve filename=/usr/lib/apt/methods/http
21775 14:26:38.453363037 0 apt (616448.616448) > execve filename=/usr/lib/apt/methods/gpgv
22335 14:26:38.461330752 0 apt (616449.616449) > execve filename=/usr/lib/apt/methods/gpgv
29434 14:26:38.481292691 0 gpgv (616451.616451) > execve filename=/usr/bin/apt-key
29604 14:26:38.486522901 0 apt-key (616453.616453) > execve filename=/usr/bin/apt-config
30183 14:26:38.494442117 0 apt-config (616454.616454) > execve filename=/usr/bin/dpkg
30422 14:26:38.497278722 0 apt-key (616455.616455) > execve filename=/usr/bin/apt-config
30996 14:26:38.504017535 0 apt-config (616456.616456) > execve filename=/usr/bin/dpkg

You can also explore the captures with Stratoshark, a GUI based on Wireshark.

Try it! 🏎️

In case you want to try out this Falco Talon 0.3.0, you can install the Helm chart following the instructions on the documentation

Let's meet 🤝

We meet every two weeks on Wednesday in our community calls, if you want to know the latest and the greatest, you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 😎

Blog: Falcosidekick 2.31.0

Tue, 04 Feb 2025 00:00:00 +0000

The year 2025 is well started now. We saw a few days ago the first release of Falco for the year. It's to let fly out a new version of Falcosidekick, the 2.31.0.

New output

This release comes with a new output only, the last pillar of the observability with [OpenTelemetry].(https://opentelemetry.io/) that missing in Falcosidekick.

OTLP Metrics

You can now forward the Falco Events to the OpenTelemetery collector or any received understanding the protocol.

New features

Here's a non exhaustive list of the great features and enhancements which come with this new release:

Better logger

It was a ToDo for a while (even years), but it's now completed. The log system used by Falcosidekick has been replaced, without any breaking change for the users, but opening the door to more enhancements in the future.

More default labels for Loki

The log lines forwarded to Loki contain now by default the source namespace and pod name, if present in the alert. It will allow to filter more easily the events you want to display in your dashboards. Thanks to @afreyermuth98.

Payload format for Loki

Some users asked for the possibility to forward the Falco alerts in their JSON format to Loki. You can now use the setting loki.format for.

NATS/STAN subject

The template for the subject where to push the messages for NATS/STAN was hardcoded, it can now be overridden with nats/stan.subjecttemplate. See the example config file.

Fixes

Fix the missing templated fields as labels in Loki payload (PR#1091)
Fix the creation error of a ClusterPolicyReport (PR#1100)
Fix the missing custom headers for HTTP requests for Loki (PR#1107 thanks to @lsroe)
Fix the wrong key format of custom fields for Prometheus (PR#1110 thanks to @rubensf)

Conclusion

You can find the full changelog here.

The respective Helm charts are already updated and allow you to test by yourself all these great new features. Just issue the helm repo update; helm upgrade --reuse-values -n falco command to do so.

Once again, thanks to all the adopters and contributors who helped and contributed to this project all these years. We would never have reached this success without you.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falco Talon project docs.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Introducing Falco 0.40.0

Tue, 28 Jan 2025 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.40.0!

This version brings several new features, performance improvements, and bug fixes that streamline Falco’s detection capabilities. During this release cycle, we merged more than 52 PRs on Falco and more than 150 PRs for libs and drivers, version 0.20.0 and version 8.0.0 respectively. Thank you to our maintainers and contributors, as this would not happen without your support and dedication!

To learn all about these changes, read on!

What’s new? TL;DR

Key features:

Streamlined Falco docker images;
Falco static build has been reintroduced for x86_64 binary using musl;
New process filters allow to filter events based on process metadata;
Added support for sendmmsg and recvmmsg syscalls parameters;
Plugins suggested output fields are now available in the Falco engine;

This release also comes with breaking changes that you should be aware of before upgrading.

Major features and improvements

The 0.40.0 release contains a number of features and UX improvements. Here is a list of some of the key new capabilities.

Streamlined Falco docker images

In the Falco 0.40.0 release, the Falco team has streamlined the Docker images to improve usability and performance. The new images are designed to be more efficient and easier to use, providing a better experience for users deploying Falco in containerized environments.

Key improvements include:

Reduced Image Size: The new images are smaller, which reduces the time required to pull and deploy them.
Optimized Layers: The layers in the Docker images have been optimized to improve build times and caching efficiency.
Enhanced Security: The images have been hardened to enhance security, reducing potential vulnerabilities.

These changes make it easier to deploy and manage Falco in various environments.

Introducing new process filters

A new set of process filters are made available in this release: proc.pgid, proc.pgid.name, proc.pgid.exe, proc.pgid.exepath, proc.is_pgid_leader. These filters enable users to filter events based on process metadata, such as the process name, executable path, and arguments. The new filters introduce the pgid field, which is directly obtained from the kernel. This ID corresponds to the host pid namespace, aiding in the creation of more reliable rules.

Plugins suggested output fields

The Falco engine now supports plugins that can suggest output fields. This feature allows plugins to provide additional context and information about an event, enhancing its visibility and understanding. The suggested output fields are displayed in the Falco output, giving users valuable insights into the event and its context. By leveraging this feature, Falco makes it easier for users to take advantage of the metadata provided by plugins and improve their security monitoring and incident response capabilities. New output fields are added only if the option is enabled and the plugin supports this new feature.

Keep an eye on the existing plugins to be updated to support the new feature.

Breaking changes and deprecations ⚠️

This version comes with breaking changes, mostly in the configuration interface.

Removed command line options and equivalent configuration options

We removed the already deprecated options --cri, --disable-cri-async, and is now possible to achieve the same result through the Falco configuration. A new configuration options has been introduced to enable and configure the supported container engines in Falco:

container_engines:
 docker:
 enabled: true
 cri:
 enabled: true
 sockets: ["/run/containerd/containerd.sock", "/run/crio/crio.sock", "/run/k3s/containerd/containerd.sock"]
 disable_async: false
 podman:
 enabled: true
 lxc:
 enabled: true
 libvirt_lxc:
 enabled: true
 bpm:
 enabled: true

Please keep in mind that the new configuration options are tagged as incubating and may change in the future.

You can also use the -o command line option:

--cri <socket_path>: use -o container_engines.cri.enabled=true, -o container_engines.cri.sockets[]=<socket_path, -o container_engines.cri.disable_async=true instead to enable the CRI runtime and configure the socket path and disable the async mode.

You can find more information on breaking changes in the tracking issue.

New docker images

With the growing prominence of the modern eBPF probe, in Falco 0.38.0 we made the strategic decision to adopt it as the default driver for Falco. This shift brings key advantages to our distribution system by removing the need to bundle the full driver-building toolchain in the standard Falco distribution. As a result, we’re transitioning the default Falco image to a no-driver/distroless configuration, simplifying deployments and reducing system complexity. For users seeking alternative setups, a different container image will still be available.

In light of this change, we’ve re-evaluated all Docker images:

Image Name	Tag (aliases)	Description
`falcosecurity/falco`	`x.y.z` (`latest`)	Distroless image without driver building toolchain support, based on the latest released tar.gz of Falco. No tools or `falcoctl` included.
`falcosecurity/falco`	`x.y.z-debian` (`latest-debian`)	Debian-based image without driver building toolchain support, based on the latest released Deb of Falco. May include some tools (e.g., `jq`, `curl`), but not `falcoctl`.
`falcosecurity/falco-driver-loader`	`x.y.z` (`latest`)	Based on `falcosecurity/falco:x.y.z-debian`, plus driver building toolchain support and the latest version of `falcoctl`. Recommended only when modern eBPF is unsupported.
`falcosecurity/falco-driver-loader`	`x.y.z-buster` (`latest`)	Similar to `falcosecurity/falco-driver-loader`, but based on a legacy Debian image (i.e., `buster`). Recommended only for old kernel versions.

The following images have been deprecated and are not anymore available in the Falco 0.40.0 release:

Image Name	Reason
`falcosecurity/falco-distroless`	Deprecated in favor of `falcosecurity/falco:x.y.z`.
`falcosecurity/falco-no-driver`	Deprecated in favor of `falcosecurity/falco:x.y.z-debian` (essentially the same image with a new name).

Deprecations

In Falco 0.40.0, we have deprecated the following options:

-S / --snaplen cli flag has been deprecated in favor of the falco_libs.snaplen configuration option;
-A cli flag has been deprecated in favor of the base_syscalls.all configuration option;
-b cli flag has been deprecated in favor of the buffer_format_base64 configuration option;

Worthy of note

Release artifacts are now built with zig, using very recent versions of clang. This change alone has resulted in up to 10% speedup in userspace benchmarks.

The first graph shows the events processed by userspace per second:

The following one shows the average of multiple runs of Google Benchmark framework embedded in libsinsp:

Additionally, artifacts now use jemalloc as the allocator library. This should help mitigate some memory fragmentation-related issues.

Furthermore, Falco debug symbol files are now attached to GitHub releases. Falco is built in RelWithDebInfo mode, enabling users to download debug symbols and attach them to their debugging sessions.

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What’s next?

Falco is more mature with each release. Following its graduation we have published the roadmap for version 1.0.0 which is guiding us in the next steps. For the next release, you can expect more stability, a new container plugin, refinements to our deployment methods with a k8s operator, and as always new detections and fixes.

Stay connected

Join us on social media and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Falcosidekick 2.30.0

Wed, 04 Dec 2024 00:00:00 +0000

A few days after a new release of Falco Talon, our response engine, it's time for our favorite proxy forwarder to do the same.

New outputs

A new release means new integrations. Thanks to our contributors for their helps.

Webex

Notify your team on Webex with the integration developed by @k0rventen.

OTLP Metrics

The adoption of Open Telemetry is bigger and bigger in the Cloud Native ecosystem, @ekoops introduced the OTLP Metrics in Falcosidekick.

Datalog Logs

The Falco alerts can be forwarded to Datadog as events for a while in Falcosidekick, you can now use their Logs service thanks to @yohboy.

New features

Here's a non exhaustive list of the great features and enhancements which come with this new release:

x3 throughput

@alekmaus spotted a bottleneck with the http client used to forward the events to the outputs. His fix increases up to 300% the throughput!!!

Better integration with Elasticsearch

@alekmaus worked hard to improve the integration with Elasticsearch. In addition improvments for the clients, new settings have been introduced, like the possibility to specify an ingest pipeline or an api key, to enable batching and compression. See the docs to know them all.

Better consistency for the Prometheus metrics

Falco recently integrated a direct endpoint to expose metrics in the Prometheus format. After a lot of discussions between the maintainers and the community, a convention has been chosen for the names of the metrics. This release adapts the metrics exposed by Falcosidekick to follow this convention and have a consistency accross the different components of the ecosystem.

Breaking changes: The renaming of the metrics might impact the queries for your alerts and dashboards.

Multi hosts for AlertManager

You can now specify a list of servers for the AlertManager output, which is a requirement when it's deployed in HA mode.

Fixes

The contributors fixed several bugs, here's a non exhaustive list of the more important ones:

Fix PolicyReports created in the same namespace than the previous event (PR#978)
Fix the missing customFields/extraFields in the Elasticsearch payload (PR#1033)
Fix the incorrect key name for CloudEvent spec attribute (PR#1051)

Conclusion

You can find the full changelog here.

Once again, thanks to all the adopters and contributors who helped and contributed to this project all these years. We would never have reached this success without you.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falco Talon project docs.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Falco Talon v0.2.0

Wed, 27 Nov 2024 00:00:00 +0000

Today we announce the release of Falco Talon 0.2.0 🦅!

Falco Talon 0.2.0 is a minor release that includes new actionners and outputs, add parameters to existing actionners, along one small fix on the check and print commands.

Features

Add gcp:function actionner:
- Now users can call GCP function to automate GCP tasks, with authentication and authorization out of the box.

- action: Invoke GCP function
 actionner: gcp:function
 additional_contexts:
 - aws
 parameters:
 gcp_function_name: simple-http-function
 gcp_function_location: us-central1

Add gcp:gcs output
- Now users can send output directly to GCP Google Cloud Storage, same way as s3 and minio existing outputs.
Add ignore_standalone_pods parameter for kubernetes:terminate actionner
Allow to wait until the completion of kubernetes:drain by configuring max_wait_period and wait_period_excluded_namespaces
Use smaller image for the kubernetes:tcpdump actionner

Fixes

An existing config.yaml file is not required anymore to check the syntax of your rules files.

Try it! 🏎️

In case you just want to try out the Falco Talon 0.2.0, you can install the helm chart following the instructions on the documentation

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest, you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 😎,

Igor

Blog: Introducing Falco 0.39.2

Thu, 21 Nov 2024 00:00:00 +0000

Today we announce the release of Falco 0.39.2 🦅!

Fixes

Falco's 0.39.2 is a small patch release that includes some important bugfixes for modern eBPF driver:

check cred field is not NULL before the access; this enables Falco back with modern eBPF driver to work on GKE
address verifier issues on kernel versions >=6.11.4: there was a kernel-breaking change in the tail call ebpf API merged into the 6.11.4 to fix a CVE. Adapt our code to work again on these new versions.

Thanks to everyone in the community for helping us spot these annoying bugs and improving Falco every day 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.39.2, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest, you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 😎,

Federico

Blog: Introducing Falco 0.39.1

Wed, 09 Oct 2024 00:00:00 +0000

Today we announce the release of Falco 0.39.1 🦅!

Fixes

Falco's 0.39.1 is a small patch release that includes some important bugfixes:

Fixed a crash when using plugin with event parsing capabilities (eg: k8smeta plugin)
Fixed a bug while parsing -o key={object} command line arguments, when the object definition contains a comma
Improved config json schema to allow null init_config for plugin info

Thanks to everyone in the community for helping us with spotting these annoying bugs and improving Falco every day 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.39.1, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 😎,

Federico

Blog: Introducing Falco 0.39.0

Tue, 01 Oct 2024 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.39.0!

This version brings several new features, performance improvements, and bug fixes that streamline Falco’s detection capabilities. During this release cycle, we merged more than 50 PRs on Falco and more than 100 PRs for libs and drivers, version 0. 18.0 and version 7.3.0 respectively. Thank you to our maintainers and contributors, as this would not happen without your support and dedication!

To learn all about these changes, read on!

What’s new? TL;DR

Key features:

Basename operator retrieves the basename of a given path;
New fields added in proc and fd classes #1916 #1936;
Regular expression operator can be used to match values in string fields;
Append output allows to add output text or fields to a subset of loaded rules;
Schema validation for config and rules files allows Falco to warn users when unknown keys are used;
Improved engine selection in Kubernetes environments driver loader will automatically pick the most compatible driver for each node in the cluster.

This release also comes with breaking changes that you should be aware of before upgrading.

Major features and improvements

The 0.39.0 release contains a number of feature and UX improvements, here are list of some of the key new capabilities.

New Operators

The basename() transformer operator extracts the base name, i.e. the filename without directory, of the input field. Note that the behavior ofbasename() in Falco is slightly different from the Unix basename program. For instance, basename (proc.exepath) will evaluate to "cat" for /usr/bin/cat but will evaluate to an empty string ("") for /usr/bin/. This allows, for instance, to write expressions like basename(proc.exepath) = cat to match against the original executable name even if it has been symlinked without knowing the full path, or any other file name based detection.

The regex operator checks if a string field matches a regular expression. Please note that the regex operator is considerably slower (up to an order of magnitude) than the above operators that work with strings, which are highly recommended for simple comparisons. The supported regex flavor is from the Google RE2 library. Example: fd.name regex [a-z]*/proc/[0-9]+/cmdline.

Introducing the Append Output Feature

In response to long-standing community requests, Falco has introduced a new feature in version 0.39.0 that allows users to add custom outputs and fields to events generated by Falco. This new functionality, called append_output, gives users greater control over the data produced by Falco rules.

With the append_output option, you can now easily add extra output to rules based on source, tag, or rule name—or even apply it to all rules without conditions. This option is configurable in the falco.yaml file and works by specifying a list of append entries, which are applied in the order they appear.

Here’s an example configuration:

append_output:
 - match:
 source: syscall
 extra_output: "on CPU %evt.cpu"
 extra_fields:
 - home_directory: "${HOME}"
 - evt.hostname

In this example, any rule with the syscall source will have the string on CPU %evt.cpu appended to the end of the default output line. Additionally, extra fields such as home_directory and evt.hostname will be visible in the JSON output under the output_fieldskey but won’t appear in the regular text output. Notably, environment variables are also supported.

This option is also available on the command line using the -o flag. For example:

falco ... -o 'append_output[]={"match": {"source": "syscall"}, "extra_fields": ["evt.hostname"], "extra_output": "on CPU %evt.cpu"}'

The introduction of append_output offers Falco users a flexible way to enrich event outputs, providing deeper visibility and customization tailored to their monitoring needs.

Dynamic Driver Selection in Falco with Helm: Simplifying Multi-Node Deployments

Deploying across diverse Kubernetes environments just got easier! When using the official Falco Helm chart and setting driver.kind=auto, the driver loader now intelligently handles the heavy lifting for you.

Here's how it works: the driver loader will automatically generate a new Falco configuration file and select the correct engine driver based on the specific node Falco is deployed on. This means whether you're using eBPF, kmod, or a modern eBPF driver, Falco will configure itself dynamically depending on the environment.

In many Kubernetes clusters, nodes can differ in terms of kernel versions, capabilities, and driver compatibility. With this new auto-selection feature, you can seamlessly deploy different Falco drivers across various nodes within the same cluster. Here’s a simple illustration:

+-------------------------------------------------------+
| Kubernetes Cluster |
| |
| +-------------------+ +-------------------+ |
| | Node 1 | | Node 2 | |
| | | | | |
| | Falco (eBPF probe) | | Falco (kmod) | |
| +-------------------+ +-------------------+ |
| |
| +-------------------+ |
| | Node 3 | |
| | | |
| | Falco (modern eBPF)| |
| +-------------------+ |
+-------------------------------------------------------+

In this example:

Node 1 is configured with the eBPF probe driver.
Node 2 uses the kmod (kernel module) driver.
Node 3 leverages the modern eBPF driver for cutting-edge performance.

This setup gives you flexibility and ensures that each node in your Kubernetes cluster is running Falco in the most optimized way possible, without manual configuration. Simply set driver.kind=auto in the Helm chart and let Falco do the rest.

Breaking changes and deprecations ⚠️

This version comes with breaking changes, mostly in the configuration interface

Removed command line options and equivalent configuration options

We removed the already deprecated options -D, -t, -T and is now possible to achieve the same result through the Falco configuration You con still use the -o command line option:

-T : use -o rules[].disable.tag=<tag> instead. Turn off any rules with a tag=<tag>. This option can be passed multiple times. This option can not be mixed with -t;
-t : use -o rules[].disable.rule=* -o rules[].enable.tag=<tag> instead. Only enable those rules with a tag=<tag>. This option can be passed multiple times;
D : use -o rules[].disable.rule=<wildcard-pattern> instead. Turn off any rules with names having the substring . This option can be passed multiple times.

You can find more information on breaking changes in the tracking issue.

Notable Bug Fixes

Prometheus Compliant metrics: some metrics have been reworked to follow the prometheus best practices #3319;
Fixed ebpf drivers to use the correct memory barrier primitive for ARM64, preventing to read incomplete data from the ring buffers #2067;
Fixed an issue where stats messages were written to stdout and could mix with regular Falco event output #3338;
fs.path fields now account for dirfd, fixing discrepancies with fd.name #1993;

Deprecations

In Falco 0.39.0, the --cri and --disable-cri-async options were deprecated, and they will be completely removed in Falco 0.40.0. Moving forward, configuring container runtimes should be done through the falco.yaml file. Below is an example of the new configuration format:

container_engines:
 docker:
 enabled: true
 cri:
 enabled: true
 sockets: ["/run/containerd/containerd.sock", "/run/crio/crio.sock", "/run/k3s/containerd/containerd.sock"]
 disable_async: false
 podman:
 enabled: true
 lxc:
 enabled: true
 libvirt_lxc:
 enabled: true
 bpm:
 enabled: true

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What’s next?

Falco is more mature with each release. Following its graduation we have published the roadmap for version 1.0.0 which is guiding us in the next steps. For the next release, you can expect more stability, streamlined container images, refinements to our rule syntax, new detections and plugins.

Stay connected

Join us on social media and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco Talon v0.1.0

Mon, 09 Sep 2024 00:00:00 +0000

More than 7 years ago, frustrated by the lack of integrations between Falco and third parties, I created Falcosidekick. The tool evolved much more than expected, with the help of dozens of contributors, individuals or for companies, to have now almost 70 different integrations, and more are coming. Its baby brother came few years later, Falcosidekick UI, helping people to visualize in real time the alerts leveraged by Falco and fine tuning their rules.

A frustation remained after all. With Falco, we have an amazing tool to detect suspicious events in our Linux hosts, VM and Kubernetes clusters, with Falcosidekick, we can easily notify our Dev/Secops, index the alerts in some SIEM, etc. But a last piece was missing: how to react to these events?

With the integrations of well known FaaS in Falcosidekick, we started a series of blog posts to show how to create from scratch what we call a "response engine". All these systems are modular, flexible, robust, but they all require a considerable amount of work from the user, to deal with the Falco payload format, the errors, the retries, the authentication to the API (AWS, Kubernetes Control Plane), the logs, the metrics, etc. Not all users and companies have the skills and/or the budget to maintain such an architecture.

To answer these needs, we designed and created Falco Talon. The the first version is officially out!.

What is Falco Talon?

Falco Talon is a Response Engine for managing threats in Kubernetes clusters. It enhances the solutions proposed by the Falco community with a no-code tailor-made solution. With easy rules, you can react to events from Falco in milliseconds.

Why did we created Falco Talon?

Over the years, the Falco community proposed different methods to react to the Falco Events, what we call a response engine.

All these methods rely on a 3rd party FaaS (Function as a Service) and come with drawbacks, all actions must be developped by the users to manage:

The errors
The Falco event format
The authentication
The K8s SDK complexity
The security
The upgrades of the dependencies
Latency
Complexity to manage sequential actions
Intrication between the function and the configuration

This is why we started to develop a custom solution specifically built for Falco: Falco Talon:

Tailor made for the Falco events
Easy to define rules
No-code implementation for end-users
UX close to Falco with the rules (yaml files with append, override mechanisms)
Allow to set up sequential actions to run
Structured logs (with a trace id)
OTEL/Prometheus Metrics and OTEL Traces

What is it good for?

React in real-time to the Falco Events
Allow fine granularity to match the events to react to
Responding to default rules with specific overrides

What is it not (yet?) good for?

Complex reaction worflows with conditions between the steps
Run actions at the host/node level through SSH (like Ansible does)

Docs

A dedicated website has been created to host the documentation: https://docs.falco-talon.org.

How Falco Talon works

As the same manner Falcosidekick works, Falco Talon receives the events from Falco by http. All you have to do to connect Falco and Falco Talon is to set in your falco.yaml:

jsonOutput: true
jsonIncludeOutputProperty: true
httpOutput:
 enabled: true
 url: "http://<falco-talon>:2803/"

If you already use Falcosidekick to forward your Falco events to the world, an integration is available since Falcosidekick 2.29.0:

talon:
 address: "http://<falco-talon>:2803/"
 checkcert: false

When the events are received by Falco Talon, an internal queue system based on NATS Jetstream is in charge to deduplicate them, to avoid to trigger the same action for the same cause for nothing.

Falco Talon will then compare the event with the rules created by the user, if an event matches with a rule, a series of actions are sequentially performed. At the end of each step, a notification with the status is sent, and a log is emmited.

Rules

The rules are the "core" of Falco Talon as they describe which actions to trigger for which Falco event.

All rules are written as yaml file, evaluated in the order they are given to Falco Talon (as arguments or in the config file), with rules specified later in the file overriding the previous ones.

The rules are composed of 2 blocks:

the action block defines which actionner to use with its parameters, this block can be imported by multiple rules (like the macros can be used in the Falco rules)
the rule block defines the criterias to match to trigger the actions

The criterias to match the event with the actions can use all elements that compose a Falco event JSON payload:

the Falco rule name
the priority
the tags
the output fields

Examples

When Falco Talon receives an event triggered by the Falco rule named Terminal shell in container, and this event doesn't concern the kubernetes namespaces kube-system and falco, then the related pod is labeled suspicious: true

- action: Label Pod as Suspicious
 description: "Add the label suspicious=true"
 actionner: "kubernetes:label"
 parameters:
 labels:
 suspicious: "true"

- rule: Terminal shell in container
 description: >
 Label the pod outside kube-system and falco namespaces if a shell is started inside
 match:
 rules:
 - Terminal shell in container
 output_fields:
 - k8s.ns.name!=kube-system, k8s.ns.name!=falco
 actions:
 - action: Label Pod as Suspicious

The action block are useful but not mandatory, the same result can be done by specifying the action in the rule block directly:

- rule: Terminal shell in container
 match:
 rules:
 - Terminal shell in container
 output_fields:
 - k8s.ns.name!=kube-system, k8s.ns.name!=falco
 actions:
 - action: Label Pod as Suspicious
 actionner: "kubernetes:label"
 parameters:
 labels:
 suspicious: "true"

Actionners

The actionners are on-catalog actions you can use. You just have to specify which one you want use to use, its parameters, and Falco Talon will manage for you all the complexity. This is how we created a no code response engine.

Within this first version, we tried to integrate as much useful actionners as possible, which allow you to manage a large variety of situations and reactions in Kubernetes.

The available actionners are:

kubernetes:terminate
kubernetes:label
kubernetes:networkpolicy
kubernetes:exec
kubernetes:script
kubernetes:log
kubernetes:delete
kubernetes:drain
kubernetes:download
kubernetes:tcpdump
aws:lambda
calico:networkpolicy
cilium:networkpolicy

To know more about what the actionners do, what parameters they require, you can read on docs/actionners.

You can notice all actionners names are composed of 2 elements x:y, the first element is the category of the actionner. All actionners in the same category share the same client, it avoid to have multi inits and instances.

Outputs

Some actionners require an output, an output is a target for the artifact created by the actionner, for example for the file retrieved by kubernetes:download or the .pcap created by kubernetes:tcpdump.

3 outputs are available today:

local:file (only useful for local tests)
aws:s3
minio:s3

The list of the available outputs can be found on docs/outputs.

Example

- rule: Redirect STDOUT/STDIN to Network Connection in Container
 match:
 rules:
 - Redirect STDOUT/STDIN to Network Connection in Container
 actions:
 - action: Run tcpdump for 5s
 actionner: "kubernetes:tcpdump"
 parameters:
 snaplen: 512
 duration: 5
 output:
 target: "aws:s3"
 parameters:
 bucket: <my-bucket>
 prefix: /tcpdump/
 region: us-east-1

Notifiers

Even we're talking about a "response engine", a framework to automatically react to some events, we still want (we humans), to be noticed of what's happening or keep traces of the performed actions.

Apart from logs output to stdout, some notifiers can be used to forward action results:

elasticsearch
k8sevents
loki
slack
smtp
webhook

The list of the available notifiers can be found on docs/notifiers.

Examples

k8sevents:

action: kubernetes:tcpdumpthought,
apiVersion: v1
eventTime: "2024-09-05T12:52:10.819462Z"
firstTimestamp: null
involvedObject:
 kind: Pod
 namespace: default
kind: Event
lastTimestamp: null
message: |
 Status: success
 Message: action
 Rule: Redirect STDOUT/STDIN to Network Connection in Container
 Action: Run tcpdump for 5s
 Actionner: kubernetes:tcpdump
 Event: Redirect STDOUT/STDIN to Network Connection in Container
 Namespace: default
 Pod: cncf-55696bc998-5xjcb
 Output: a tcpdump "tcpdump.pcap" has been created
 TraceID: c954bd8b3391a08f23079552fdd639f3
metadata:
 creationTimestamp: "2024-09-05T12:52:10Z"
 generateName: falco-talon-
 name: falco-talon-zgxfm
 namespace: default
 resourceVersion: "115862544"
 uid: 3b4bd17f-ed1a-4693-bfd7-d10f674a8008
reason: falco-talon:action:kubernetes:tcpdump:success
reportingComponent: falcosecurity.org/falco-talon
reportingInstance: falco-talon
source:
 component: falco-talon
type: Normal

slack:

A tool designed for the production

I spent 10 years of my career as a DevOps/SRE, managing traditional and cloud infrastructures, I know how painful it is to manage systems not well designed for the runtime. This is why we tried from the beginning to create a tool easy to rule all along it lifecycle.

A CLI to validate the rules

As it is for the Falco rules, the best way to manage the lifecycle of the rules for Falco Talon is to follow the GitOps principles. This requires to set up a validation of their syntax as step in the CI.

The Falco Talon binary can also be used as a CLI, allowing to perfom tasks on the rules, like checking their validity or printing their results after the merges/overrides of several files:

$ falco-talon rules check --help

Check Falco Talon Rules file

Usage:
 falco-talon rules check [flags]

Flags:
 -h, --help help for check

Global Flags:
 -c, --config string Falco Talon Config File (default "/etc/falco-talon/config.yaml")
 -r, --rules stringArray Falco Talon Rules File (default [/etc/falco-talon/rules.yaml])

Examples

With a valid rules file:

$ falco-talon rules check -c ./config.yaml -r ./rules.yaml

2024-09-05T16:42:28+02:00 INF rules result="rules file valid"

With an invalid rules file:

$ falco-talon rules check -c ./config.yaml -r ./rules.yaml

2024-09-05T16:44:01+02:00 ERR rules error="unknown actionner" action="Label Pod as Suspicious" actionner=foor:bar rule="Terminal shell in container"
2024-09-05T16:44:01+02:00 FTL rules error="invalid rules"
exit status 1

Structured Logs

The logs, whatever the component emitting them, keep always the same structure and contain a trace_id field, allowing to follow the workflow performed by Falco Talon.

The value of trace_id is also used to create the TraceId the OTEL Traces, by using a log backend like Loki, it becomes easy to correlate the traces with the logs in the same UI, like Grafana.

The CLI contains more features, take a look at them on docs /installation_usage/usage.

Example

Each step is clearly identified by the tag after the log level:

2024-09-05T14:52:03+02:00 INF event event="Redirect STDOUT/STDIN to Network Connection in Container" output=<truncated> priority=Critical source=syscall trace_id=c954bd8b3391a08f23079552fdd639f3

2024-09-05T14:52:03+02:00 INF match event="Redirect STDOUT/STDIN to Network Connection in Container" output=<truncated> priority=Critical rule="Reverse shell detected" source=syscall trace_id=c954bd8b3391a08f23079552fdd639f3

2024-09-05T14:52:10+02:00 INF action action="Run tcpdump for 5s" actionner=kubernetes:tcpdump event=test namespace=default output="a tcpdump 'tcpdump.pcap' has been created" pod=cncf-55696bc998-5xjcb rule="Reverse shell detected" status=success trace_id=c954bd8b3391a08f23079552fdd639f3

2024-09-05T14:52:10+02:00 INF notification action="Run tcpdump for 5s" actionner=kubernetes:tcpdump notifier=k8sevents rule="Reverse shell detected" stage=action status=success trace_id=c954bd8b3391a08f23079552fdd639f3

2024-09-05T14:52:11+02:00 INF output action="Run tcpdump for 5s" bucket=xxxxx category=aws file=tcpdump.pcap key=2024-09-05T14-52-10Z_default_cncf-55696bc998-5xjcb_tcpdump.pcap output="the file 'tcpdump.pcap' has been uploaded as the key 'tcpdump/2024-09-05T14-52-10Z_default_cncf-55696bc998-5xjcb_tcpdump.pcap' to the bucket 'xxxxx'" output_target=aws:s3 prefix=tcpdump/ region=us-east-1 status=success trace_id=c954bd8b3391a08f23079552fdd639f3

2024-09-05T14:52:11+02:00 INF notification action="Run tcpdump for 5s" actionner=kubernetes:tcpdump notifier=k8sevents output_target=aws:s3 rule="Reverse shell detected" stage=output status=success trace_id=c954bd8b3391a08f23079552fdd639f3

Metrics

Falco Talon exposes the traditional /metrics endpoint with metrics in the Prometheus format.

To keep a consistency, all metrics related to Falco Talon itself are prefixed with falcosecurity_falco_talon_, it follows the same convention used by Falco for its metrics.

For people interested by the metrics in the OTEL format, it's also available, see docs installation_usage/metrics

Example

# HELP action_total number of actions
# TYPE action_total counter
falcosecurity_falco_talon_action_total{action="Disable outbound connections",actionner="kubernetes:networkpolicy",event="Test logs",namespace="falco",otel_scope_name="github.com/Falco-Talon/falco-talon",otel_scope_version="devel",pod="falco-5b7kc",rule="Suspicious outbound connection",status="failure"} 6
falcosecurity_falco_talon_action_total{action="Terminate Pod",actionner="kubernetes:terminate",event="Test logs",namespace="falco",otel_scope_name="github.com/Falco-Talon/falco-talon",otel_scope_version="devel",pod="falco-5b7kc",rule="Suspicious outbound connection",status="failure"} 6
# HELP event_total number of received events
# TYPE event_total counter
falcosecurity_falco_talon_event_total{event="Unexpected outbound connection destination",otel_scope_name="github.com/Falco-Talon/falco-talon",otel_scope_version="devel",priority="Critical",source="syscalls"} 2
# HELP match_total number of matched events
# TYPE match_total counter
falcosecurity_falco_talon_match_total{event="Unexpected outbound connection destination",otel_scope_name="github.com/Falco-Talon/falco-talon",otel_scope_version="devel",priority="Critical",rule="Suspicious outbound connection",source="syscalls"} 2

OTEL Traces

We know following logs can be not really convenient, and they may lack of useful informations. You can therefore enable the emits of Traces in the OTEL format. All backends accepting this format can be used to store and visualize them.

To know how to set up the traces, see docs installation_usage/traces.

Examples

In Grafana with Tempo:

In Jaeger:

Installation

The easiest way, for now, to deploy Falco Talon is to use the Helm chart included in the repo.

with Helm

Since version 0.2.0, chart has been moved under the official falcosecurity/charts repository

The procedure to install the v0.1.0 of Falco Talon is:

helm repo add falcosecurity https://falcosecurity.github.io/charts
helm repo update falcosecurity
helm upgrade --install falco-talon falcosecurity/falco-talon

Shoutout

I would like to shoutout some persons without the project would have been possible:

Dan Papandrea who thought about the first specs of the project with me and found the name Falco Talon
Igor Eulalio who develops Falco Talon with me, introduced amazing features like the traces, and injected so much energy in the project
Rachid Zarouali, the tester #1, a lot of features came from his ideas and feedbacks, he's also always a pleasure to present a talk about Falco Talon with him
Nigel Douglas who tests and promotes Falco Talon with talks and blog posts since the alpha stages
Carlos Tadeu Panato Júnior the magician of the CI, who still continue to manage the upgrade of the dependencies

What's next?

This first release, the v0.1.0, is just GA and it's the beginning of the journey. All your feebacks and ideas are welcome, this project has for DNA to improve the security of the Kubernetes clusters by answering real needs and usages.

The next big step to achieve is to join officially the falcosecurity organization. An issue has been created in the evolution repo to do so. Don't hesitate to vote for 🙏!

Thomas

To go further:

GitHub repo of the Falco Talon project: https://github.com/falco-talon/falco-talon
Official docs of Falco Talon: https://docs.falco-talon.org/
A record of a talk (by Rachid and Thomas) in French to introduce Falco Talon: https://www.youtube.com/watch?v=Mx28fhyKX7Q

Blog: Introducing Falco 0.38.2

Mon, 19 Aug 2024 00:00:00 +0000

Today we announce the release of Falco 0.38.2 🦅!

Fixes

Falco's 0.38.2 is a patch release that includes the most important bugfixes addressed this summer ☀️:

Fixed a crash when using transformer operators (e.g. tolower()) with a parameter that evaluates to an empty string
Fixed a bug and a regression that could result in incorrect comparison between ipv4 addresses and ipv6 subnets and vice versa
Fixed an issue that could result in missing exe_upper_layer flag
Fixed kernel module build for Linux 6.10
Fixed a bug that may result in kernel module crashes on recent versions of RHEL 9
Added additional logging to better troubleshoot hard to reproduce issues like "could not parse param ... for event ... of type ...: expected length X, found Y"

This patch also introduces a small change with the format of the new experimental Prometheus metrics to make them easier to use. Metrics are now distinguished by the file_name or rule_name labels, in line with Prometheus best practices and supporting groupBy queries.

Thanks to everyone in the community for helping us with spotting these annoying bugs and improving Falco every day 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.38.2, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 😎,

Luca

Blog: Falcosidekick 2.29.0

Tue, 02 Jul 2024 00:00:00 +0000

Almost 1 year without a release of Falcosidekick, but version 2.29.0 is finally here. Thanks to all contributors for their patience, you made amazing contributions and we're happy to finally have them available for all users.

Like for every releases, a small recap about its adoption. Falcosidekick continues to be adopted, even if the rate is not as high as before, but we're sure it will explode once again with this new fresh version.

Once more, Falcosidekick expands Falco's integrability with a lot of new outputs. That and the introduction of many new features has been possible thanks to the hard work of the community. You can find a comprehensive list of these in the changelog.

New outputs

More and more systems are integrated as outputs in Falcosidekick, more and more often directly by the companies themselves and not their end users. It shows Falco and Falcosidekick are seen as major components in the security fields, and trusted as de facto standards.

Dynatrace

Mario Kahlhofer, aka @blu3r4y, from Dynatrace, integrated the well known observability and security platform he works for. You can even read his blog post about, to discover how to correlate the Falco events with their APM agent events.

Sumologic

Carlo Mencarelli, aka @mencarellic, did the exporter of the Falco events to Sumologic, the SaaS platform for your logs.

OTLP Traces

It started as an internal hackaton at Grafana Labs and became a real integration thanks to JuanJo Ciarlante (@jjo). You can now export the Falco event as traces, to have an automatic correlation between the detected events.

[!WARNING] It works only for the syscall related events.

Quickwit

After a demo of Falco at a CNCF Meetup, the Quickwit team wanted to add their product as a new output for Falcosidekick, and they did it. You can now easily index your Falco events in their search engine thanks to the work of Idriss Neumann (@idrissneumann).

Falco Talon

New born in the Falco ecosystem, trying to complete the last missing piece: the reaction. You can now forward the Falco events to Falco Talon, a tailor made no-code response engine for Falco. The project is still in alpha stage, but moves quickly. Stay tuned.

New features

Aside from new outputs, we introduced very important and useful new features. Let's do a recap of them.

Revamp of the Policy Report output

The Policy Report feature in Kubernetes evolved since its integration in Falcosidekick, it was the time to do some clean up. The report now contains more information, and their displays in the Policy Reporter UI is better.

New outputFieldFormat setting

Some systems perform deduplication of the events, for example the on-call platforms. They use the content of the output to do so, but the current format starting with a timestamp prevents the process to run as expected. A new setting outputFieldFormat is now available allows to "format" the output field of the Falco payload before forwarding it to the outputs.

The default format received from Falco is : <timestamp>: <priority> <output> which corresponds to this:

14:37:27.505989596: Warning Detected ptrace PTRACE_ATTACH attempt (proc_pcmdline=%proc.pcmdline evt_type=%evt.type user=%user.name user_uid=%user.uid user_loginuid=%user.loginuid process=%proc.name proc_exepath=%proc.exepath parent=%proc.pname command=%proc.cmdline terminal=%proc.tty exe_flags=%evt.arg.flags %container.info)

By removing the <timestamp> and <priority>, you get:

Detected ptrace PTRACE_ATTACH attempt (proc_pcmdline=%proc.pcmdline evt_type=%evt.type user=%user.name user_uid=%user.uid user_loginuid=%user.loginuid process=%proc.name proc_exepath=%proc.exepath parent=%proc.pname command=%proc.cmdline terminal=%proc.tty exe_flags=%evt.arg.flags %container.info)

If you use the settings customFields and templatedFields of Falcosidekick to inject new elements in the output_fields, it's also possible to have them in the output with the tokens <custom_fields> and <templated_fields>.

Alternative endpoints for AWS S3

Some projects like Minio are S3-compliant, you can now use them as target for the AWS S3 output by changing the endpoint to use. Thanks to @gysel for this feature.

Split of the docs

The main README of the project became really huge over the years, with all those available outputs. We did a big refactor and you can now find one file per output, with more details about the configuration, the default values and some tips. The docs are here, and any help is welcome to make them even better.

Fixes

The contributors fixed several bugs, here's a non exhaustive list of the more important ones:

Fix missing root CA for the Kafka output (thanks to @claviola)
Fix bug with the extension source in the CloudEvent output
Fix panics in the Prometheus output when hostname field is missing
Fix locks in the Loki output (thanks to @bsod90)
Fix mTLS client verification failures due to missing ClientCAs (thanks to @jgmartinez)
Fix wrong env vars for pagerduty output
Remove hard settings for usernames in Mattermost and Rocketchat
Fix multi lines json in the error lines (thanks to @idrissneumann)
Fix duplicated custom headers in clients
Fix the labels for the AlertManager output (thanks to @Umaaz)

Conclusion

You can find the full changelog here.

Once again, thanks to all the adopters and contributors who helped and contributed to this project all these years. We would never have reached this success without you.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falco Talon project docs.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Introducing Falco 0.38.1

Wed, 19 Jun 2024 00:00:00 +0000

Today we announce the release of Falco 0.38.1 🦅!

Fixes

Falco's 0.38.1 is a patch release aimed at addressing a few important bugs. It includes the following fixes:

A Falco crash while running with plugins and metrics enabled has been solved (https://github.com/falcosecurity/falco/issues/3229)
Falco -p output format option can now be passed to plugin events while -pc and -pk can only be used for syscall sources. Fixes an issue that could result in Falco exiting with LOAD_ERR_COMPILE_OUTPUT on startup with k8s clusters that had -pk and audit enabled (https://github.com/falcosecurity/falco/pull/3239)
Fixed an issue that could prevent the integer compare operators <, <=, >, >= in rules from working properly (https://github.com/falcosecurity/falco/issues/3245)
Ignore NSS user and group entries while loading users and groups (https://github.com/falcosecurity/libs/pull/1909)
Issues related to the new metric-related plugins API (https://github.com/falcosecurity/libs/pull/1885). Plugin API was also bumped to 3.6.0.
Plugin metrics are now enabled in Falco (https://github.com/falcosecurity/falco/pull/3228). Note that plugin must make use of the new metrics-related API to expose metrics.
Libs were updated to 0.17.2

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.38.1, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Federico

Blog: Introducing Falco 0.38.0

Thu, 30 May 2024 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.38.0! This is the first Falco release since its graduation within the CNCF, and, as usual, brings many improvements and features alongside some pretty big changes in its configuration mechanism.

This release brings an easier to use mechanism to install and configure your drivers, new rule language features, better support for Falco metrics and many more improvements.

During this release cycle, we merged more than 100 PRs on Falco and more than 180 PRs for libs and drivers, version 0.17.0 and version 7.2.0 respectively. Thank you to our maintainers and contributors, as this would not happen without your support and dedication!

To learn all about these changes, read on!

What’s new? TL;DR

Key features:

New capabilities in falcoctl to automatically select the best driver for your system and make it easier to install
The Falco configuration file can now be split into multiple files to make it more manageable
Rule selection from configuration file or command line
Field transformers and value comparison
Prometheus metrics support
Plugin API improvements

This release also comes with breaking changes that you should be aware of before upgrading.

Major features and improvements

The 0.38.0 release contains a number of feature and UX improvements, here are list of some of the key new capabilities.

Driver loader magic ✨

If we could pick the most common issue that we've heard from adopters and we experienced first hand is the fact that sometimes we all struggle with installing and upgrading Falco drivers. The Falco team has been tirelessly working for years to improve the installation experience and Linux kernel compatibility with massive changes such as the introduction of the new CO-RE eBPF probe and most recently the complete rewrite of our driver loading component, integrated in falcoctl. With this new version of falcoctl, integrated in Falco 0.38.0, our loading tool will automatically detect your system and pick the most compatible driver without any intervention; on recent kernel versions this is likely the modern eBPF probe. As you probably know, the modern probe does not require any extra driver download or compilation, making it load almost instantly. Of course, the tool also allows to select the preferred driver if the automatic choice is not optimal for your use case. On top of that, our driver loader tool can now automatically download kernel headers for many distributions supported by driverkit so in many cases you will be able to install even the kernel module without having to install kernel headers first. Read more about how to configure this functionality in the installation documentation page.

Organize your Falco configuration files 🗃️

Our falco.yaml configuration file gains more options, fine tuning configuration flags and feature selection for every release; in fact, they are so many that some people would like to better organize them in separate configuration files which can also be kept across Falco upgrades. Starting from this release you can add list of files or directory to the config_files configuration entry, which comes populated with the /etc/falco/config.d/ directory by default. Any additional file is read in order and can override settings in falco.yaml. Read more in the configuration options section of the documentation.

Choose which rules to load at runtime 📝

We distribute several files that contain community contributed rules and you can always write your own. But how do you select which rules Falco will load at runtime? There are several ways, including using overrides or specifying command line options such as -D, -t and -T. However, those do not allow you to express something as simple as "I would like to exclude all rules except for this one" or "I would like to include a specific tag and disable some of its rules". Furthermore, you couldn't specify this configuration in your falco.yaml file. To make this possible, we introduced a new configuration option, rules, that can be specified both in the configuration file or the command line. For instance, you can now write:

rules:
 - disable:
 rule: "*"
 - enable:
 rule: Netcat Remote Code Execution in Container
 - enable:
 rule: Delete or rename shell history

To finely control your rule loading without modifying the rule files themselves. Read more in controlling rules.

Field transformers and value comparison in conditions

Up until now we couldn't write a condition that catches operations like "a process deleting its own executable" because you couldn't use a field value on the right hand side of the condition. Since this version we have added a syntax to do just that with the val() operator:

evt.type = unlink and proc.exepath = val(fs.path.name)

will trigger only if the process exepath is the same as the unlink argument target, meaning that the process is trying to delete its own executable!

In addition you can also apply simple transform operators to both sides of the comparison: toupper() and tolower() will convert casing as you'd expect and b64() can decode base64. Stay tuned for additional transformers to cover more use cases! Read more on transform operators in the documentation.

Prometheus Metrics support 🔥

If you have been following Falco development, you probably know we are constantly improving support for metrics that tell you how the Falco engine is doing. We now have introduced Prometheus support so you can better integrate Falco with your existing performance monitoring infrastructure, and paves the way for the community to create an official Grafana dashboard that can be integrated in our charts.

Plugin API improvements ⚙️

Plugins are getting more powerful at each version. We now have a set of experimental APIs to expose metrics and read more into the Falco internal state that our expert plugin authors have been asking about. Stay tuned for more in-depth documentation on those!

Breaking changes and deprecations ⚠️

This version comes with breaking changes, mostly in the configuration interface

Changed configuration options

The syscall_buf_size_preset Falco configuration option has been replaced by engine.<driver>.buf_size_preset (e.g. engine.kmod.buf_size_preset)
The syscall_drop_failed_exit Falco configuration option has been replaced by engine.<driver>.drop_failed_exit (e.g. engine.kmod.drop_failed_exit)
The modern_bpf.cpus_for_each_syscall_buffer Falco configuration option has been replaced by engine.modern_ebpf.cpus_for_each_buffer
The syscall_event_drops Falco configuration option has been replaced by the metrics config plus some automatic notification on drops.

Removed command line options and equivalent configuration options

The --modern_ebpf command line option has been replaced by engine.kind: modern_ebpf in falco.yaml (or, on the command line -o engine.kind=modern_ebpf). Likewise, --nodriver is now engine.kind: nodriver.

The environment variable FALCO_BPF_PROBE is replaced by engine.ebpf.probe configuration option. Example:

engine:
 kind: ebpf
 ebpf:
 # path to the elf file to load.
 probe: ${HOME}/.falco/falco-bpf.o

The -e option to load capture files is no longer available. In order to read a capture file use the configuration option engine.replay.capture_file. Since options can be specified on both the command line and the configuration file, an equivalent command line as falco -e <file.scap> is falco -o engine.kind=replay -o engine.replay.capture_file=<file.scap>

The gVisor command line options have been replaced by equivalent configuration options. -g/--gvisor-config is now engine.gvisor.config while --gvisor-root is now engine.gvisor.root. Example falco.yaml configuration file:

engine:
 kind: gvisor
 gvisor:
 # A Falco-compatible configuration file can be generated with
 # '--gvisor-generate-config' and utilized for both runsc and Falco.
 config: "/etc/docker/runsc_falco_config.json"
 # Set gVisor root directory for storage of container state when used
 # in conjunction with 'gvisor.config'. The 'gvisor.root' to be passed
 # is the one usually passed to 'runsc --root' flag.
 root: "/var/run/docker/runtime-runc/moby"

Or, equivalent writing on the command line:

falco -o engine.kind=gvisor -o engine.gvisor.config=/etc/docker/runsc_falco_config.json -o engine.gvisor.root=/var/run/docker/runtime-runc/moby

You can find more information on breaking changes in the tracking issue.

Deprecations

In Falco 0.39.0 we will remove the -D, -t, -T options, continuing our tradition of removing single-character options that nobody remembers what they do.

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What’s next?

Falco is more mature with each release. Following its graduation we have published the roadmap for version 1.0.0 which is guiding us in the next steps. As you can see, this version is addressing some of the roadmap points with our changes to configuration and CLI options and adding rule constructs and drivers. For the next release, you can expect more stability, streamlined container images, refinements to our rule syntax, new detections and more.

Stay connected

Join us on social media and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco 0.37.1

Tue, 13 Feb 2024 00:00:00 +0000

Today we announce the release of Falco 0.37.1 🦅!

Fixes

Falco's 0.37.1 release is a small patch aimed at addressing a few minor bugs. It includes the following:

Added --http-insecure flag to driver loader images
Added new env variable FALCOCTL_DRIVER_HTTP_HEADERS understood by driver loader images to pass a comma separated list of http headers for driver download, eg: FALCOCTL_DRIVER_HTTP_HEADERS='x-emc-namespace: default,Proxy-Authenticate: Basic'
Falcoctl was bumped to v0.7.2, fixing an issue building Flatcar drivers and a bug withing the kernel release fixup method to build drivers download URLs
Fixed a nasty bug that caused Falco to crash when a priority higher than debug was set in the config: https://github.com/falcosecurity/falco/pull/3060
Libs were updated to 0.14.3

Last, but not least, as recommended by the CNCF, we now link libelf dynamically instead of statically, so that the library remains separable from Falco at runtime.
This has multiple outcomes:

Falco static (musl) build is disabled for now; we are experimenting with some solutions and we will hopefully be able to bring it back up soon
Users of docker images won't notice anything since they already shipped libelf library
Users of deb and rpm packages won't notice anything since libelf was already a nested dependency
Users of the tar.gz package will need to manually install libelf where not present

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.37.1, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Federico

Blog: Introducing Falco 0.37.0

Tue, 30 Jan 2024 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.37.0!

This release brings an improved installation experience, a new way to modify Falco rules, and some great UX improvements. There are, as to be expected, a handful of breaking changes. But, rest assured, we've done all we can to help you with any changes you might need to make.

During this release cycle, we merged more than 100 PRs on Falco and more than 160 PRs for libs and drivers, version 0.14.2 and version 7.0.0 respectively. Thank you to our maintainers and contributors, as this would not happen without your support and dedication!

To learn all about these changes, read on!

What’s new? TL;DR

Key features:

A new way to enrich syscalls with Kubernetes metadata, replacing the old Kubernetes collector.
New capabilities in falcoctl to download and build our kernel drivers, replacing the old falco-driver-loader script.
Support for 32-bit syscall emulation on x86_64 in all kernel drivers (modern_ebpf, ebpf, kernel module).
A new override key to easily modify rules, lists, and macros.

Key UX improvements:

Introduction of a new engine key in falco.yaml to replace all other methods for opening engines such as FALCO_BPF_PROBE, --modern-bpf, -g, and -e.
Expansion of environment variables in falco.yaml even when they are part of a string.

This release also comes with breaking changes, we'd suggest to read them before upgrading. If you use helm, make sure to read the Helm chart breaking changes page as well.

Major features and improvements

The 0.37.0 release contains a number of feature and UX improvements, here are list of some of the key new capabilities.

A new way to enrich syscalls with K8s metadata

Falco 0.37.0 introduces a new method to enrich syscalls with Kubernetes metadata to help address scalability and other issues with the old collector. Falco always had Kubernetes support, but sometimes we need new approaches to keep up with the bigger and bigger scale that we see in production clusters today. You can find more technical details here.

While the collector was previously integrated into Falco, this feature uses a new architecture which leverages a plugin (k8smeta) and a remote collector (k8s-metacollector).

The plugin gathers details about Kubernetes resources from the remote collector. It stores this information and provides access to Falco upon request. The plugin specifically acquires data for the node where the associated Falco instance is deployed, resulting in node-level granularity. In contrast, the collector runs at the cluster level.

Within a given cluster there may be multiple k8smeta plugins (one per node), but only one collector exists per cluster.

More technical details about the architecture and design choices are here.

It’s important to note that both new components are considered experimental, which means although they are functional and tested, they are currently in active development. They may undergo changes in behavior as necessary without prioritizing backward compatibility.

Fields supported by the new `k8smeta` plugin

This section provides details on:

Kubernetes fields that are supported out-of-the-box by Falco through container runtime enrichment.
Fields the new k8smeta plugin supports
Fields have been deprecated.

The following fields are automatically populated with data from the container runtime, making them compatible with Falco without needing the old k8s collector or the new k8smeta plugin. These fields will continue to function as before, and no changes have been made:

k8s.pod.name
k8s.pod.id/k8s.pod.uid
k8s.pod.sandbox_id
k8s.pod.full_sandbox_id
k8s.pod.label
k8s.pod.labels
k8s.pod.ip
k8s.pod.cni.json
k8s.ns.name

All other fields with the k8s.* prefix previously supported by the old collector (e.g., k8s.deployment.name) are now deprecated and will return <NA> if used in rules.

These fields are now provided by the new plugin under the k8smeta.* prefix. A complete list of these fields can be found here

The new fields introduced by the k8smeta plugin are additive. They do not replace the fields provided by the container runtime. This means you can use both k8s.pod.name and k8smeta.pod.name simultaneously. While they may return the same value, the data is collected from different sources (container runtime for k8s fields, the Kubernetes API server for k8smeta). As a result, their availability and reliability may differ during the lifecycle of an application. While it may seem redundant, this approach should offer flexibility to users.

To wrap up:

If k8s.pod.* and k8s.ns.name fields meet your needs, you can use Falco without plugins. The default container runtime information in Falco should be enough.
If k8s.pod.* and k8s.ns.name fields are insufficient, you should evaluate the new k8smeta plugin.
The old k8s.* fields (excluding k8s.pod.* and k8s.ns.name) are now deprecated, and if used in Falco rules, they return <NA>

If you’d like to read more about this new feature check out the documentation for the k8smeta pluginand the k8s-metacollector, while if you want to deploy this solution with our helm chart check out the dedicated section.

New Falcoctl capabilities

Since falcoctl 0.7.0, users have been able to quickly download and compile Falco drivers using the falcoctl driver command. Starting with Falco 0.37.0 the falcoctl driver command will be used by the Falco installation process in place of the old falco-driver-loader script.

For example, to install the kernel module:.

Specify which driver we want to use

falcoctl driver config --type kmod

Install the driver

falcoctl driver install

By default, the falcoctl driver install command tries to download a prebuilt driver from the official Falco download s3 bucket. If a driver is found, then it is inserted into ${HOME}/.falco/. Otherwise, the script tries to compile the driver locally.

You can find more details on installing each driver type in our docs.

Finally, while the falcoctl driver command replaces the old falco-driver-loader script it’s important to note that, even though there is no change in terms of usage, the Docker images falco-driver-loader and falco-driver-loader-legacy no longer utilize the old falco-driver-loader script; instead, they now use falcoctl.

32-bit syscall emulation

The support for 32-bit syscalls has consistently been a highly requested feature for a long time. Until now, this support was only available in the kernel module, but starting from Falco 0.37.0, we have finally extended this support to the ebpf and modern_ebpf drivers. This feature is crucial as it addresses a security gap that has existed for some time.

It’s important to note that this feature is specifically for 32-bits syscalls emulated on the x86_64 architecture. Falco does not support pure 32-bit architectures.

Follow these steps to try out this new feature:

Create a C program ia32.c

#include <stdio.h>
#include <unistd.h>
#include <sys/syscall.h>
#include <sys/types.h>

int main() {
 syscall(__NR_close, -1);
 return 0;
}

Compile it gcc ia32.c -o ia32 -m32
Start Falco with the following rule evt.type = close and proc.name contains ia32
Execute the binary

./ia32

You should see the rule triggered

New override key

Falco 0.37.0 replaces the append: true key-value pair with a new override section. The override section allows you to either replace or append keys to a rule, macro, or list value . It’s important to note that you cannot append and replace the same key; you must choose one or the other. Choosing both will result in an error.

The keys that can be modified vary according to the rules component being overridden. See the override documentation for the full list of keys that can be modified.

The override section can either be in a custom rules file or can be in the same file as the component being overridden. In either case, the override section needs to be specified after the rule that is being modified. When the override is in the same file, the override section needs to be below the original rule, list, or macro definition. If the override is in another file, that file needs to be loaded after the original rules file.

A quick example from the documentation illustrates how this new feature works.

In this example, the original rule is in falco_rules.yaml and the override is specified in falco_rules.local.yaml.

/etc/falco/falco_rules.yaml

- rule: program_accesses_file
 desc: track whenever a set of programs opens a file
 condition: proc.name in (cat, ls) and evt.type=open
 output: a tracked program opened a file (user=%user.name command=%proc.cmdline file=%fd.name)
 priority: INFO

/etc/falco/falco_rules.local.yaml

- rule: program_accesses_file
 condition: and not user.name=root
 output: A file (user=%user.name command=%proc.cmdline file=%fd.name) was opened by a monitored program
 override:
 condition: append
 output: replace

The modified program_accesses_filerule would trigger when ls or cat use open on a file, unless they were run by root.

The new output message would be A file (user=%user.name command=%proc.cmdline file=%fd.name) was opened by a monitored program

A final note, the old append: true method of modifying values has been deprecated and will be removed in Falco 1.0.0.

Additional UX improvements

Introduce a new unique engine key in falco.yaml to replace all the other methods of opening engines (FALCO_BPF_PROBE, --modern-bpf, -g, -e). See the deprecated features section for more info.
Falco now expands environment variables in falco.yaml even when they are part of a string. It is now possible to use syntax similar to this:

ebpf:
 probe: ${HOME}/.falco/falco-bpf.o

Our gVisor integration has also been improved by adding support for more events, including write, socketpair, timerfd_create and an updated configuration generator. In addition, we added support for any gVisor container ID format, making Falco more robust and compatible with gVisor sandboxed containers beyond Docker and Kubernetes.

Breaking changes

This is a list of breaking changes introduced in Falco 0.37.0

The Rate-limiter mechanism was removed as it is no longer used.
--userspace CLI option was removed as it’s no longer used.
The falco-driver-loader script is removed and embedded into falcoctl
The Helm chart 4.0.0 contains several modifications to work with the new k8s metadata collector. Please read its breaking change file for more information.
The new falcoctl driver implementation will drop:
- --source-only
- BPF_USE_LOCAL_KERNEL_SOURCES
- DRIVER_CURL_OPTIONS
- The FALCO_BPF_PROBE environment variable won't be used by the new falcoctl driver loader as it is already deprecated and scheduled to be removed in the next major version.
Various environment variables have been replaced as part of the new falcoctl driver feature:
- DRIVERS_REPO has been replaced by FALCOCTL_DRIVER_NAME or the --name command line argument.
- DRIVERS_NAME has been replaced by FALCOCTL_DRIVER_REPOS or the --repo command line argument.
- DRIVER_KERNEL_RELEASE has been replaced by --kernelrelease command line argument.
- DRIVER_KERNEL_VERSION has been replaced by --kernelversion command line argument.
- DRIVER_INSECURE_DOWNLOAD has been replaced by --http-insecure command line argument.
Remove -K/-k options from Falco in favor of the new k8s plugin
Dropped plugins shipped with Falco since plugins will now be managed by falcoctl. If you want to use a plugin like k8saudit be sure to install it at init time with falcoctl.
A new feature in Falco 0.37.0 allows environment variables to be expanded even if they are part of a string. This new functionality introduces a minor breaking change.

Previously, environment variables used in YAML that were empty or defined as “” would be expanded to the default value. This was inconsistent with how YAML was handled in other cases, where we only returned the default values if the node was not defined.

With Falco 0.37.0 we will return the default value for nodes that cannot be parsed to the chosen type. The program_output command will be environment-expanded at init time instead of letting popen; thus, the shell expands it.

This is technically a breaking change, even if no behavioral change is expected.

Note that you can avoid environment var expansion by using ${{FOO}} instead of ${FOO}. It will resolve to ${FOO} and won't be resolved to the environment var value.

You can find more information on breaking changes in the tracking issue.

Deprecated features

This is a list of features that will be removed in Falco 0.38.0

Modern probe Docker builder is no longer used.
syscall_buf_size_preset Falco config in favor of engine.kmod/ebpf/modern_ebpf.buf_size_preset.
syscall_drop_failed_exit Falco config in favor of engine.kmod/ebpf/modern_ebpf.drop_failed_exit.
modern_bpf.cpus_for_each_syscall_bufferFalco config in favor of engine.modern_ebpf.cpus_for_each_buffer.
FALCO_BPF_PROBE environment variable in favor of engine.ebpf.probe.
-e command line flag in favor of engine.replay.capture_file.
g,gvisor-config command line flag in favor of engine.gvisor.config.
gvisor-root command line flag in favor of engine.gvisor.root.
modern-bpf command line flag in favor of engine.kind=modern_ebpf.
nodriver command line flag in favor of engine.kind=nodriver.
syscall_event_drops falco config will be replaced by the metrics config plus some automatic notification on drops.

Be sure to check the tracker issue for more information.

Try it out

Interested in trying out the new features? Use the resources below to get started.

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What’s next?

The community is active on many fronts, and we plan on delivering more great features and stability fixes during the next release cycle!

Some of the things we are currently working on include:

Implement further improvements to our rule framework and rule syntax.
Add new features and enhancements to falcoctl to make it even more powerful.
Enhance the quantity, quality, and presentation of metrics in Falco.

And much much more

Stay connected

Join us on social media and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to hearing your feedback and ideas.

You can find all the most up-to-date information at https://falco.org/community/.

Blog: Introducing Falco 0.36.2

Fri, 27 Oct 2023 00:00:00 +0000

Today we announce the release of Falco 0.36.2 🦅!

Fixes

Falco's 0.36.2 release is a small patch addressing a few bugs. It includes the following:

Fixed a possible segfault caused by uninitialized variable in libsinsp::next() method call. (https://github.com/falcosecurity/falco/issues/2878)
Improved supported program type detection for modern BPF; this ensures we can actually be sure that our BPF program type is unsupported when returning an error to the user. (https://github.com/falcosecurity/libs/pull/1404)
Fixed a subtle bug in rawarg filtercheck for non-string types. (https://github.com/falcosecurity/libs/pull/1428)
Fixed an uninitialized variable in the libscap bpf engine that lead to stdin getting closed while Falco soft restarted. (https://github.com/falcosecurity/libs/issues/1448)

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.36.2, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What next? 🔮

The community is active on many topics and we hope to deliver great features and many stability fixes once again during the next release cycle!

The old falco-driver-loader script is showing its age and it's time to work on a more maintainable solution. falcoctl is a great candidate to host everything driver related, implement new features and make our lives easier when we need to install Falco drivers on a new machine.
Our rule framework is brand new and we forsee many improvements and active development work on it.
The latest Falco versions brought many improvements to the plugin framework; we wish to use those to create a more scalable Kubernetes client plugin that will be able to withstand much heavier loads and will be easier to maintain.

And many, many, more enhancements!

To get a weekly reminder of all the great stuff happening in the Falco lands, make sure to join the #falco channel on the Kubernetes Slack!

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions:

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Federico

Blog: Introducing Falco 0.36.1

Mon, 16 Oct 2023 00:00:00 +0000

Today we announce the release of Falco 0.36.1 🦅!

Fixes

Falco's 0.36.1 release is a small patch aimed at protecting our uses by addressing a few minor bugs. It includes the following:

Address a HIGH severity vulnerability in libcurl CVE-2023-38545, bumping the library to the patched version 8.4.0. You can find more details in the section below.
The legacy eBPF probe can now handle systems with CPU hotplug enabled, opening the right number of kernel buffers. (https://github.com/falcosecurity/falco/issues/2843)
Remove a no longer useful experimental Falco config outputs_queue.recovery. This was introduced in Falco 0.36.0 as an experiment.
Fix a possible segfault caused by a faulty implementation of timer_delete. (https://github.com/falcosecurity/falco/issues/2850)

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Vulnerability in libcurl

A HIGH severity vulnerability in libcurl, CVE-2023-38545, was disclosed alongside a patched version (8.4.0). We would like to answer the main question you might have about it: Does it affect Falco?

According to the excellent in-depth description of the bug, this can only be triggered if both conditions below are true:

A SOCKS5 HTTP(S) proxy has been configured. This happens if you have set the standard environment variables that control proxy connections, such as http_proxy/https_proxy/no_proxy or libcurl-specific ones as indicated in the advisory or the libcurl documentation.
An attacker controls the server that Falco is connecting to, namely the server configured to receive http_output or a custom prebuilt driver repository server, and the SOCKS5 proxy is "slow enough" to allow the attack to happen.

While it may be rare that users have an exploitable environment, it's still a possibility. For this reason, Falco maintainers decided to ship this patch release 🦅

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.36.1, you can install its packages following the process outlined in the docs:

Prefer to use a container image? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What next? 🔮

The community is active on many topics and we hope to deliver great features and many stability fixes once again during the next release cycle!

The old falco-driver-loader script is showing its age and it's time to work on a more maintainable solution. falcoctl is a great candidate to host everything driver related, implement new features and make our lives easier when we need to install Falco drivers on a new machine.
Lately we have expanded the syscall coverage that Falco can provide. We wish to improve these efforts across all drivers with even more 32 bit syscalls.
Our rule framework is brand new and we forsee many improvements and active development work on it.
The latest Falco versions brought many improvements to the plugin framework; we wish to use those to create a more scalable Kubernetes client plugin that will be able to withstand much heavier loads and will be easier to maintain.

And many, many, more enhancements!

Let's meet 🤝

We meet every Wednesday in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Andrea, Luca

Blog: Falco 0.36.0

Tue, 26 Sep 2023 00:00:00 +0000

Dear Falco Community, today we are happy to announce the release of Falco 0.36.0!

This releases comes as usual with many new features and improvements. Thanks to everyone that worked on all the features, bugfixes and improvements! To read a detailed account of the release, see v0.36.0 in the changelog.

During this release cycle, we merged more than 100 PRs on Falco and more than 150 PRs for libs and drivers, version 0.13.1 and version 6.0.1 respectively. Thank you to our maintainers and contributors, as this would not happen without your support and dedication!

This release comes with many brand new features, some long awaited UX improvements and configuration and also beware of some breaking changes! Don't worry, everything is explained below!

What's new? TL;DR

In release v0.36.0, we focused on the following features:

Brand new Falco rule framework and ruleset
More robust executable file path detection, symlink resolution and ancestors detection
Falco is no longer limited to one rule firing per event!
Signatures are now automatically verified in Falcoctl for plugins and rules
Upgrade of the default Falco images

We have also some massive experimental upgrades that the community has spent incredible amounts of effort on:

WASM support
Kernel driver testing at scale!
Falco now has an experimental distroless container image based on Wolfi

Breaking changes ⚠️

We have seen many requests from the community in the form of questions and issues. Those are the ones that shape the evolution of Falco, so we can hopefully make the user experience better at every release. Sometimes, in order to do this we need to implement changes that may impact some workflows. In this release we have important breaking changes you should be aware of:

The default rules file that is shipped in the Falco image and/or can be downloaded via falcoctl as falco-rules is now a stable rule file. This file contains a much smaller number of rules that are less noisy and have been vetted by the community. This serves as a much requested "starter" Falco rule set that covers many common use case. The rest of that file has been expanded and split into falco-incubating-rules and falco-sandbox-rules. Read more below to learn about the difference.
The main falcosecurity/falco container image and its falco-driver-loader counterpart have been upgraded. Now they are able to compile the kernel module or classic eBPF probe for relatively newer version of the kernel (5.x and above) while we no longer ship toolchains to compile the kernel module for older versions in the default images. Downloading of prebuilt drivers and the modern eBPF will work exactly like before. The older image, meant for compatibility with older kernels (4.x and below), is currently retained as falcosecurity/falco-driver-loader-legacy.
The Falco HTTP output no longer logs to stdout by default for performance reasons. You can set stdout logging preferences and restore the previous behavior with the configuration option http_output.echo in falco.yaml.
The --list-syscall-events command line option has been replaced by --list-events which prints all supported system events (syscall, tracepoints, metaevents, internal plugin events) in addition to extra information about flags.
The semantics of proc.exepath have changed. Now that field contains the executable path on disk even if the binary was launched from a symbolic link.
The -d daemonize option has been removed.
The -p option is now changed:
- when only -pc is set Falco will print container_id=%container.id container_image=%container.image.repository container_image_tag=%container.image.tag container_name=%container.name
- when -pk is set it will print as above, but with k8s_ns=%k8s.ns.name k8s_pod_name=%k8s.pod.name appended
Command line options s and stats-interval have been removed in favor of metrics config in falco.yaml.

Major features and improvements

New Falco rules framework 🛡️

This project is the result of a discussions that started a long time ago and required a massive amount of work from the community. Following this proposal we have decided to split the rules that the Falco community maintains into three main groups, described in the maturity levels section of the contributing guide:

Stable Falco rules. Those are the only ones that are bundled in the Falco by default. It is very important to have a set of stable rules vetted by the community. To learn more about the criterias that are required for a rule to become stable, see the contributing guide
Incubating rules, which provide a certain level of robustness guarantee but have been identified by experts as catering to more specific use cases, which may or may not be relevant for each adopter.
Sandbox rules, which are more experimental.

It is important to keep in mind that the stable ruleset is significantly changed since the last release! Not only the rules are a much smaller subset but they have been refined and they may have been renamed according to the style guide.

Thanks to Melissa Kilby for driving this effort 🚀!

The list of releases for each type of rule is present in the repository, where you can download each file. They can also be downloaded from the download page and are also available as signed OCI artifacts for download via falcoctl!

Want to contribute to the rules? You can find more information in the contribution guide and the style guide.

Process executable and lineage 🪪

We have achieved a higher level of accuracy and data quality regarding the existing proc.exepath field and the process tree reconstruction in general. This step forward reinforces our commitment to refining Falco and providing you with an even better user experience.

In more detail:

The proc.exepath process executable path field now contains a resolved version of the executable path, meaning that even if an executable was launched from a symlink, the field will show the original location of the binary. In the past, we resolved the exe argument in userspace by utilizing the process's cwd when the path was not absolute. Conversely, if exe was absolute, the exepath was equivalent to exe. The new implementation ensures the extraction of the authentic and accurate disk path of the executable when it resides on the disk.
As it turns out, it's not that simple to reconstruct the complete process tree in a Linux system. The Linux kernel presents intriguing edge case behaviors, where the direct parent process might genuinely have already exited. In the past, Falco encountered difficulties in continuing to reconstruct the parent process lineage in such situations. To address this, we've enhanced Falco's logging capabilities. Now, even in scenarios where the parent process has exited, Falco can continue reconstructing the process tree.

Container image changes 📦

We have two big changes to our default container images:

The falco-driver-loader image is now based on Debian Bookworm with a more modern version of compilers, meaning that it will be much easier to build on contemporary systems but you might see compilation issues for older kernels (4.x and below). For that, the falco-driver-loader-legacy image is provided! Also, this means that vulnerability scanners will not report so many false positive vulnerabilities in the new version of the image since it does not contain legacy versions of compilers.
We have a falco-distroless image based on Wolfi, thanks to contributions from Adrian Mouat and the Falco Supply Chain Security WG! This is for all of you that are fans of minimal images! You can try it out by replacing falco-no-driver with falco-distroless.

Falcoctl ❤️ cosign

Since Falco 0.35.0 we started providing signed official container images signed with cosign in keyless mode. But how about our other OCI artifacts, which are rules and plugins? Starting from Falcoctl 0.6.1, shipped with this release, all of the official rules and plugins are signed and automatically verified at installation time thanks to the magic of cosign in keyless mode!

Thanks to Massimiliano Giovagnoli for his help along with the Falco Supply Chain Security WG! Stay tuned for an in-depth explanation of the security architecture of this feature.

Multiple rules can be matched on each event

Pro Falco users know that we could only match one rule for each event. This is not true anymore, and since this version we have a rule_matching option in the configuration file. rule_matching: all will remove this limitation and match everything. See the documentation in falco.yaml for more information!

Big experimental contributions

Last but not least, we have several big projects that we have started with the community and are very proud of.

Falco Kernel Testing Framework

Falco supports a large number of Linux kernels. And the truth is, in order to test this kind of functionality you have to start an (ideally) equally large number of live Linux systems and load the driver there. This is absolutely not easy to do and just taking a look at the task list for such an endeavor gives you an idea of the complexity required. The results are awesome: you can find a matrix of kernels that are continuously tested for x86_64 and ARM as well! See the in-depth blog post to learn much more about this!

Falco WASM

Flaco is excited to introduce its latest addition: the WebAssembly target. This new target has been developed exclusively for the Falco Playground using Emscripten, where it brings essential core functionalities to the forefront. These functionalities include a rule compiler and the ability to reproduce events from capture files. It’s worth noting that certain features, such as kernel modules and Kubernetes support, have been intentionally omitted from this wasm target. This omission is due to the inherent limitations of running these features within a web browser environment. falco.wasm can be found as a github artifact in the latest workflow.

Falco Playground

Falco playground is simple web application where you can create, edit and validate falco rules. This is a quick solution for users wanting to easily check the accuracy of their custom rules. This application is completely client side and doesn’t make calls to any backend server. It leverages the power of WebAssembly to test your rules. You can try it live and find the code in the falco-playground repository!

Additional UX improvements

With each release, Falco gets more quality-of-life improvements, such as:

Environment variables resolution in configuration files
A new outputs_queue configuration option to better fine tune Falco's output performance

Deprecated features

It's sad to see features go, but sometimes we need to remove something in order to focus on what matters for our adopters. This is what maintainers are proposing for deprecation in this release and removal in the next Falco version 0.37.0:

The optional rate-limiter mechanism, since it seems to be no longer used and it also can discard events including potentially critical alerts
The --userspace option, since the corresponding feature and the associated projects in the Falco organization have not been maintained for years
The falco-driver-loader bash script. The driver loading functionality is going to be implemented in falcoctl to improve Falco's driver loading capabilities and make it easier to maintain and contribute to.

Try it out

It’s time to try out the new release! Here are some pointers for getting started with Falco:

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What next? 🔮

The community is active on many topics and we hope to deliver great features and many stability fixes once again during the next release cycle!

The old falco-driver-loader script is showing its age and it's time to work on a more maintainable solution. falcoctl is a great candidate to host everything driver related, implement new features and make our lives easier when we need to install Falco drivers on a new machine.
Lately we have expanded the syscall coverage that Falco can provide. We wish to improve these efforts across all drivers with even more 32 bit syscalls.
Our rule framework is brand new and we forsee many improvements and active development work on it.
The latest Falco versions brought many improvements to the plugin framework; we wish to use those to create a more scalable Kubernetes client plugin that will be able to withstand much heavier loads and will be easier to maintain.

And many, many, more enhancements!

Stay tuned 🤗

Join us in our communication channels and in our weekly community calls! It’s always great to have new members in the community, and we’re looking forward to having your feedback and hearing your ideas.

You can find all the most up to date information at https://falco.org/community/.

See you for the next release!

Enjoy,

Luca, Andrea, Rohith

Blog: Falcosidekick-UI 2.2.0

Thu, 14 Sep 2023 00:00:00 +0000

Not so long ago, we proudly released a new fantastic release of falcosidekick, it's time for its little brother, falcosidekick-ui to know the same, with the version v2.2.0.

Let's take a tour to introduce the most important cool new features of this release.

Disabling the authentication

The previous version introduced a basic auth mechanism to protect access to the dashboard and API. Some complained it broke the access through their reverse proxy. You can now disable the authentication:

-d boolean
 Disable authentication (environment "FALCOSIDEKICK_UI_DISABLEAUTH")

Dialog box to display the details of an event

To have a better view of each event, you can now open a dialog box that displays all details but also the raw JSON of the event. You can even copy it into your clipboard with a simple click.

To display the dialog box, just click on the {...} at the end of the event row.

Export

A new Export button appeared, it allows you to export all the events found in json format. It takes in consideration the filters, of course.

Units for TTL

For users with a lot of events, it can be useful to specify a TTL (time to live) for the keys in Redis (the storage backend). It can be done with -t argument for a while, and the value had to be in seconds, which is not convenient for long-term storage. You can now specify a unit (seconds, minutes, hours, Weeks, Months, year). If no unit is specified, it's considered as seconds to avoid breaking previous configs.

-t string
 TTL for keys, the format is X<unit>,
 with unit (s, m, h, d, W, M, y)" (default "0", environment "FALCOSIDEKICK_UI_TTL")

Redis password

The access to the dashboard and the API can be protected by credentials, but the Redis wasn't. You can now specify a password for access to Redis, it will prevent your security scans from complaining ;-).

-w string
 Redis password (default "", environment "FALCOSIDEKICK_REDIS_PASSWORD")

Conclusion

Thanks again to our amazing community, most of these features came from your ideas and we're still thrilled to see how much you find falcosidekick-ui useful.

As usual, if you have any feedback or need help, you can find us at any of the following locations.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falcosidekick UI project on GitHub.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Falcosidekick 2.28.0

Fri, 28 Jul 2023 00:00:00 +0000

It's summertime, it's hot, and many people are on vacation, but the Falco community is still there. Six months after the release of Falcosidekick's latest upgrade, version 2.28.0 becomes officially available.

The number of pulls of the official Falcosidekick image from Docker Hub has also just reached 15M, which we consider mind-blowing. It took it 3 years to reach the first 5M pulls and now it's needed less than six months to do it again. Awesome!

New outputs

With every new release, the number of integrations of Falcosidekick increases. We have reached the number of 58 available outputs already, and we hope the threshold of 60 will be left behind very soon.

Redis

Redis is a well-known in-memory database with many years of adoption on its path. It is now possible for Falcosidekick to use it as an output destination, thanks to the contributions of pandyamarut.

Telegram, the instant messaging platform, is becoming more and more used by companies for notifications, and thanks to zufardhiyaulhaq, it can receive Falco alerts too.

N8N

Do you want to extend the possibilities or just avoid developing a script to react to Falco events? Here comes n8n.

Grafana OnCall

At the last KubeCon, we met with some of the Grafana maintainers. We discussed the integration of Falco using Falcosidekick within the OnCall project. It's done now.

OpenObserve

OpenObserve is a young but promising full stack observability platform.

New features

Aside from new outputs, we introduced very important and useful new features. Let's do a recap of them.

Use different methods for the Webhook output

Since its implementation, the Webhook output has only used the HTTP method POST. Now, you can choose between the POST and PUT methods to send your data, extending the catalog of possible REST APIs to use it with.

webhook:
 method: "POST" # HTTP method: POST or PUT (default: POST)

Replace the brackets in the payload

Some Falco fields refer to lists and reflect that their keys contain brackets, like proc.args[0], proc.args[1], etc. Unfortunately, some outputs may refuse payloads with brackets in keys. For this reason, we introduced the possibility of replacing them with any other chosen character:

bracketreplacer: "_" # if not empty, the brackets in keys of Output Fields are replaced

Set custom headers for Loki, Elasticsearch and Grafana outputs

If you want to protect your private instances of Loki, Grafana, or Elasticsearch you may need to specify custom headers. This new feature allows you to do so.

elasticsearch:
 customHeaders: # Custom headers to add in POST. Useful for Authentication
 key: value

Match the priority with the severity for the AlertManager output

AlertManager is a pretty common software at companies also using Prometheus. Until now, the mapping between the Priority of Falco events and the Severity of AlertManager was already predefined. You can now define it depending on your needs thanks to Lowaiz.

alertmanager:
 customseveritymap: "" # comma separated list of tuple composed of a ':' separated Falco priority and Alertmanager severity that is used to override the severity label associated to the priority level of falco event. Example: debug:value_1,critical:value2. Default mapping: emergency:critical,alert:critical,critical:critical,error:warning,warning:warning,notice:information,informational:information,debug:information. (default: "")

Set thresholds for the dropped events for the AlertManager output

Another contribution from Lowaiz: You can now configure a set of thresholds to start dropping the events.

alertmanager:
 # dropeventdefaultpriority: "" # default priority of dropped events, values are emergency|alert|critical|error|warning|notice|informational|debug (default: "critical")
 dropeventthresholds: # comma separated list of priority re-evaluation thresholds of dropped events composed of a ':' separated integer threshold and string priority. Example: `10000:critical, 100:warning, 1:informational` (default: `"10000:critical, 1000:critical, 100:critical, 10:warning, 1:warning"`)

Better integration with TimescaleDB

We thank hileef for improving the integration with TimescaleDB.

User roleARN and externalID for the AWS outputs

Under some situations, you may want Falcosidekick to assume a role, possibly from another account. You can do it with the new pair of settings rolearn and externalid:

aws:
 rolearn: "" # aws role to assume (optional if you use EC2 Instance Profile)
 externalid: "" # external id for the role to assume (optional if you use EC2 Instance Profile)

Set the region for the PagerDuty output

Falcosidekick allows you to select between one of the two different regions' PagerDuty offers now.

pagerduty:
 region: "us" # Pagerduty Region, can be 'us' or 'eu' (default: us)

Allow TLS for the SMTP output

It is now possible to communicate with an SMTP server using TLS.

smtp:
 tls: false # Use TLS connection (true/false). Default: true

Set attributes to GCP PubSUb messages

GCP PubSub accepts attributes in its messages. You can specify yours, thanks to annadorottya.

gcp:
 customAttributes: # Custom attributes to add to the Pub/Sub messages
 key: value

More options for TLS and mTLS

These are the most relevant changes of this release. To improve security, Falcosidekick can now listen using HTTPS with TLS. You can also be more specific with the keys and certificates for the mTLS for the outputs.

mutualtlsfilespath: "/etc/certs" # folder which will used to store client.crt, client.key and ca.crt files for mutual tls for outputs, will be deprecated in the future (default: "/etc/certs")
 # certfile: "/etc/certs/client/client.crt" # client certification file
 keyfile: "/etc/certs/client/client.key" # client key
 cacertfile: "/etc/certs/client/ca.crt" # for server certification
tlsserver:
 # certfile: "/etc/certs/server/server.crt" # server certification file
 keyfile: "/etc/certs/server/server.key" # server key

The mutualtlsfilespath setting is kept for now for backward compatibility but it will be remove in future

In some edge cases, you may need some endpoints to listen in HTTP only. You can specifically define them together with the associated port:

tlsserver:
 notlsport: 2810 # port to serve http server serving selected endpoints (default: 2810)
 notlspaths: # if not empty, a separate http server will be deployed for the specified endpoints
 - "/metrics"
 - "/healthz"

Thanks to annadorottya for her impressive work on this functionality.

Autocreate the topic for the Kafka output

When Falcosidekick doesn't detect the topic, it can create it automatically. This feature is not enabled by default.

kafka:
 topiccreation: false # auto create the topic if it doesn't exist (default: false)

Support multiple bootstrap servers and TLS for the Kafka output

To get better resiliency, you can now specify several bootstrap servers and even communicate with them with TLS, thanks to ibice.

kafka:
 tls: false # Use TLS for the connections (default: false)

Fixes

We're not going to go into detail about all the corrections made in this version - you can find the full list here. Thanks to everyone who reported issues and to those who have corrected them.

The most important have been:

Fix breaking brackets in AWS SNS messages.
Fix setting name for the table of TimescaleDB output (thanks to alika).
Fix the cardinality issue with Prometheus labels.
Fix panic when asserting output fields that are nil.
Fix URL generation for Spyderbat output (thanks to bc-sb).
Fix nil values in Spyderbat output (thanks to spider-guy).
Fix duplicated headers in SMTP output (thanks to apsega).

Conclusion

The respective Helm charts are also updated and allow you to test for yourself all these great new features. Just issue the helm repo update; helm upgrade --reuse-values -n falco command to do so.

Falcosidekick is now mentioned in the official Falco docs. It's a shy beginning, but more details will come shortly.

Once again, thanks to all the adopters and contributors who helped and contributed to this project all these years. We would never have reached this success without you.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falcosidekick UI project on GitHub.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Falco 0.35.1

Thu, 29 Jun 2023 00:00:00 +0000

Today we announce the release of Falco 0.35.1 🦅!

Novelties 🆕 and Fixes

Here is a tiny patch release! It addresses some small bugs that will not bother us and our users anymore:

Bug fix in the plugin framework, we can now associate a thread ID also to async events so that we can access related juicy information when writing rules! We suggest updating to this version to be able to use all the new capabilities that the new Plugin API has to offer!
Modern BPF can now be used in least privileged mode without any trouble in COS
Driver loader now correctly parses the kernel version of Ubuntu’s kernel flavors, and also supports Debian rt and cloud
Solved a rule ordering problem on our default ruleset that caused some rules to be shadowed
Updated falcoctl to the latest version, which fixes a corner cases that cause the tool to freeze

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.35.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

We are in the working to let new big things happen in Falco, stay tuned!

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Jason and Lorenzo

Blog: Falco 0.35.0

Wed, 07 Jun 2023 00:00:00 +0000

Dear Community, today we are delighted to announce the release of Falco 0.35.0!

A big thank you to all our contributors for helping get the latest release out, we are thrilled to share this release and its goodies with the community. To read a detailed account of the release, see v0.35.0 in the changelog.

During this release cycle, we had 90+ PRs on Falco and a grand total of 170+ PRs for libs 0.11.0 and 60+ for drivers 5.0.1. Thank you to our maintainers and contributors, as this would not happen without your support, dedication, and contribution!

What's new? TL;DR 🩳

In release v0.35.0, we focused on addressing the following key features:

Moving the modern eBPF probe out of experimental status
Improving Falco performance, allowing tailoring syscall detection to one's needs
New Falco metrics
Falco images signing
Improving plugins SDK
Test infra revamp

For more information check out the 0.35 overview video on YouTube

Modern eBPF probe 👨‍🚀

The new, modern eBPF probe was released as experimental during the 0.34.0 release cycle. Since then we worked hard to implement all the remaining syscalls and behaviors, and now the same eBPF probe has left experimental status.

The new eBPF probe is a CO-RE probe, which means it is already built into Falco, and you don't need any downloads. Moreover, it sports better performance compared to the old eBPF probe.

Finally, while delivering the new eBPF probe, Andrea Terzolo also shipped a brand new driver testing framework, now used in libs CI to test consistency between all three drivers. This addition alone was worth the effort: on behalf of the whole community, thank you Andrea!

The new probe has stricter kernel release requirements: for more info, check out our blog post.

Improved Falco performance

Thanks to the collaborative effort from Melissa Kilby, Jason Dellaluce, Andrea Terzolo and Federico Di Pierro, we were able to completely revamp the way that Falco detects syscalls that needs to be captured. With the new adaptive syscalls feature, Falco will only enable syscalls that are needed to detect the ruleset it is being run with. It will also enable a bunch of syscalls that are needed for libsinsp internal state parsers, and that's it.

Consequently, the -A flag semantics have changed. By default, ie. without -A, heavy syscalls (like I/O ones) won't be captured, even if the ruleset ships with them, and a warning is shown to the user. Using -A will now allow Falco to capture even heavy syscalls, without showing a warning. A couple of new config keys are now available to further tailor Falco adaptive syscalls: a related blog post will be published soon, so stay tuned!

One of the neatest things about this work is that the huge libs refactor it required lays the groundwork for another highly requested feature: LSM and kprobes support.

Falco metrics

Thanks to yet another collaborative effort led by Melissa, Falco has a new experimental metrics feature. This introduces a redesigned stats/metrics system, emitted as monotonic counters at predefined intervals (Prometheus-like).

There are multiple options available: one can enable the output of these metrics as internal metric snapshot rule, allowing them to be emitted as outputs. Or you can choose to output metrics to a file, that is not rotated by Falco. Moreover, there are options to enable CPU and memory usage metrics, internal kernel event counters and libbpf stats.

This is a great first step to improve Falco resource observability!

Falco images signing

Starting from 0.35.0, all Falco images that you can deploy in your cluster are now signed with cosign 2.0 in keyless mode.
This means that you can always verify that the Falco image you downloaded is an official Falco image, regardless of which registry you downloaded it from. Moreover, you don't have to install or explicitly trust any public key for it to work. This is the magic of cosign in action!

So, how do you verify our brand new images? Install cosign 2 and run:

cosign verify docker.io/falcosecurity/falco:0.35.0 \
--certificate-oidc-issuer=https://token.actions.githubusercontent.com \
--certificate-identity-regexp=https://github.com/falcosecurity/falco/ \
--certificate-github-workflow-ref=refs/tags/0.35.0

Of course, you can do the same for all the deployable images including falco, falco-driver-loader, falco-no-driver and falcoctl (see its repo for more details).

This wouldn't have been possible without a big effort from Luca Guerra and Federico Di Pierro to migrate our entire release pipeline from CircleCI to GitHub Actions. The work is part of a larger effort from the Falco Supply Chain Working Group to bring all the Falco official artifacts up to date with the latest supply chain security standards. Special thanks to Massimiliano Giovagnoli, Batuhan Apaydın and Carlos Panato for your help and expertise in this area!

Plugins workstream

The Plugin API has seen quite a few big improvements, mainly from Jason.

The first big change is that the plugin framework is now totally compatible with all the events supported by the Falco libraries, including all system calls and kernel events. The plugin API now shares all the event definitions of libscap and allows plugins to both produce syscall events and extract fields from them. This feature has been in big demand since the first plugin system release (#410, #992), and opens the door to many new opportunities for Falco extensions.

Second, plugins now have a standard way for managing and maintaining internal state. Up until now, plugins were only able to extract fields from the information available in the payloads of each event, thus being stateless components by definition. Now, plugins have a defined protocol (#991) for hooking into the event stream, reconstructing an internal state, and using it for extracting fields for Falco rules. Also, plugins can inject asynchronous metadata events in open data streams to notify about state transitions and make them reproduceable when replaying capture files, just like has always happened with container-related events in the Falco libraries.

Lastly, plugins are now able to communicate bidirectionally with the Falco libraries and access their internal state, both in read and write modes. For example, this enables creating plugins that extract metadata fields from syscall event streams, and that have access to all the thread information reconstructed by libsinsp, with the opportunity of enriching it dynamically at runtime. The API surface also allows cross-plugin state access. We hope the developer community will appreciate the new power this offers plugin authors.

This big feature package required altering the plugin API in a way that is incompatible with the previous versions (the API major version has been bumped). As such, plugins released after Falco version 0.35 will not be compatible with Falco versions <= 0.34.1, and plugins released before version 0.35 will not be compatible with Falco from version 0.35 onwards. So, the action required for you is to remember to also update all your plugins to the latest versions when updating Falco to v0.35!

Test-infra revamp

Massimiliamo Giovagnoli and Samuele Cappellin have contributed tremendous work on improving our infra. Prow is now lighter, quicker and less issue-prone. Multiple prow jobs were moved to GitHub Actions to improve cluster efficiency; moreover, driver-building jobs are now much less frequently killed (basically never).

Also, arm64 drivers are now built on arm64 nodes, without using qemu, speeding up the build time. At the same time, resources allocated to the cluster were enlarged, with autoscaling limits now set to 20 ARM nodes and 20 x86 nodes. We can now deliver weekly new driver artifacts much quicker than before!

Finally, the cluster now exposes Grafana dashboards for monitoring purposes: https://monitoring.prow.falco.org/.

Try it out

It’s time to try out the new release! Here are some pointers for getting started with Falco:

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

What next? 🔮

The community is active on many topics and we hope to deliver great features and many stability fixes once again during the next release cycle!

We will revisit and improve libsinsp API, for a more coherent developer experience.
Finally, the long-awaited LSM and kprobes will be implemented.
As the plugin API has seen huge improvements, we expect new plugins using the new features very soon.
Fixes, fixes and also fixes everywhere
Above all, we will work to improve thread tables and process trees inconsistencies; that's a huge topic and we plan to tackle it in multiple ways!

Stay tuned 🤗

You can find all the most up to date information at https://falco.org/community/.

See you for the next release!

Federico, Andrea and Lorenzo

Blog: Falco 0.34.1

Mon, 20 Feb 2023 00:00:00 +0000

Today we announce the release of Falco 0.34.1 🦅!

Novelties 🆕 and Fixes

Here's a minor update! This patch release addresses small but persistent issues that have been causing inconvenience for users:

http_output not working as expected when the remote endpoint was using the HTTPS protocol;
FALCO_ENGINE_VERSION was bumped since in Falco 0.34.0 new event fields were added for the process events;
cleanups and fixes related to memory management were introduced in libs;
avoid file descriptor leakage when checking for online CPUs in libpman.

Thanks to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.34.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

It's an exciting time for Falco as we see so many great improvements and features. What's more exciting is the fact that many great ideas and awesome work are going on to make the next big things happen.

The upcoming release will include complete syscall support in the modern BPF probe (feature parity with kernel module and current BPF probe) and introduce adaptive syscall selection for the Falco ruleset.

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Aldo

Blog: Falco 0.34.0 a.k.a. "The Honeybee 🍯"

Tue, 07 Feb 2023 00:00:00 +0000

Dear community, today we are delighted to announce the release of Falco 0.34.0 🎉!

A big thank you to the community for helping get the latest release out. The Falco community is thrilled about this release and cannot wait to share the latest goodies. Check out the newest features from this most recent launch to learn more ⬇️. To read a more detailed account of the release, see v0.34.0 in the changelog.

What’s new? 🆕

In this release we saw more than 190 pull requests across the repos of Falco and its libraries. Thank you to our maintainers and contributors, as this would not happen without your support, dedication, and contribution.

Updates - TL;DR 🩳

In release v0.34.0 the community focused on addressing the following key features:

Downloading and dynamically updating Falco rules at runtime
Shipping the brand new experimental modern eBPF probe
Designing more ways to catch suspicious executions

Automatic rules update 🔄

A few questions that often come up when using Falco is how can we update rules once Falco is installed in the cluster and, how do we get updated rules from the Falco organization without having to wait for the next release? This is the first release to include falcoctl with an out of the box solution to do exactly that!

When using the new Falco Helm Chart 3.0.0 rules will automatically be updated from the official repository. To learn more about this feature and how to configure it read the Helm chart documentation. Likewise, when using a SystemD based install you can configure Falco to automatically update rules.

Want to upgrade to the new Helm chart? Read all you need to know before you do so!

Modern eBPF probe 👨‍🚀

Last quarter, Andrea published the blog, “Getting started with modern BPF probe in Falco,” and announced that the new experimental eBPF probe had landed among us bringing with it a few key features: CO-RE paradigm, BPF Ring Buffer map, BTF-enabled program, BPF global variables, BPF skeleton, and finally Multi-arch support.

Why a new probe? 👽

The old probe supported old kernels (>=4.14) that can not take advantage of the new shiny eBPF features. While it would be great to have only one probe that works for every kernel version, recent features change (and simplify!) the way we write, maintain and deploy the code so deeply that a new fresh probe is the most reasonable solution. In order to leverage these recent eBPF improvements and use the new probe you will need a kernel version >= 5.8.

Modern eBPF in action 🎬

Try it now!

Shiny new eBPF features ✨

Why are Falco maintainers and community members excited about the modern eBPF probe? There are quite a few features that you might be interested in! Some of our favorites are:

CO-RE paradigm - stands for "Compile-once-run-everywhere", so as you may imagine, this paradigm allows compiling the eBPF probe just once for all kernels! You understood well: NO MORE MISSING DRIVERS, and no more painful local builds requiring the much-loved KERNEL HEADERS.
Multi-arch support - the modern BPF probe also supports multiple architectures by design. The actual targets for Falco are x86_64 and arm64 but new ones can be added at any time. If you have a project that needs BPF instrumentation for one of these architectures you could simply link the Falco libraries (libsinsp, libscap) to obtain a working solution out of the box.
Performance Improvements - the modern eBPF probe leverages features recently introduced in the Linux kernel such as BPF global variables and ring buffers to be faster and more efficient than the traditional eBPF probe!

Even more ways of catching suspicious executions 🕵️‍♀️

Detecting when a suspicious new executable is spawned is often considered a crucial baseline detection. Generally speaking, detecting this kind of behavior and understanding when it is malicious is not an easy task. For this reason Falco has not one, but several features that can help defenders craft appropriate rules for their workflows.

Thanks to great contributions from Lorenzo Susini and Melissa Kilby (thanks for both the code contributions and the image above!) we have two more ways to check for suspicious executions in our Falco rules as we have the following new fields tied to process spawn (execve) events:

proc.is_exe_upper_layer: which is true if the process’ executable is in the upper layer of the overlayfs. In practice, that means that the executable that is being launched has been introduced or modified in the container after it was started. While some applications might do this legitimately, in many cases it is a thing to watch out for because it might be signaling an attack in progress! Note that you can use this only on kernel versions greater or equal than 3.18.0, since overlayfs did not exist before then, and of course with container runtimes that make use of it as a union mount filesystem 😉. This flag complements proc.is_exe_writable, which is similar but only checks if the executable file is also writable by the same user that spawned it.

Don’t think this is enough? Do you think you need more flags to get more accurate detections? Here’s the second group of fields:

proc.exe_ino.ctime and proc.exe_ino.mtime: they show the last change time and modification time of the process’ executable file, respectively.
proc.exe_ino.ctime_duration_proc_start and proc.exe_ino.ctime_duration_pidns_start: demonstrates the time difference, in nanoseconds, between the process ctime and when the process was actually spawned or when the PID namespace was created, respectively. I’m sure you can see why you could be interested in that. Launching executables that were just created could be something that you want to know about 😁.

While the above signals won't replace the need to monitor file operation events, they can help reduce the search space for tracking spawned processes where for example chmod +x was run against the executable file on disk prior to execution (this causes ctime of inode to change, but we don't know if it was chmod related or a different status change operation). In addition, users could use these fields for selected rules to augment information available for incident response.

Artifact distribution 📜

Automatic rules updates and other upcoming features would not be possible without a proposal aimed to create a unified management of the distribution of artifacts. The overall goals for this are:

Allow users to consume artifacts in a consistent way
Define official artifacts
Unify distribution mechanism, infrastructure, and tooling
Provide generic guidelines applicable to any artifact that is distributed

The officially supported artifacts are a set of artifacts published by Falcosecurity and now are part of Falco and its ecosystem. Prior to release 0.34.0 the Falcosecurity organization distributed several kinds of artifacts in the form of files or container images, which included:

Installation packages
Helm charts
Drivers (eg, kmod, eBPF)
Rule files
Plugins
Other kinds may be added in the future.

Now, the new distribution channels include HTTP Distribution and OCI Distribution.

What we accomplished ✅

Falco rules have their own repo now 🏠

The benefits of having rules living in their repository are:

Dedicated versioning
Rules release will not be tied anymore to a Falco release (e.g., no need to wait for the scheduled Falco release to publish a new rule aiming to detect the latest published CVE)
Consistent installation and update mechanism for other rulesets (plugins rules are already published in their repository and can be consumed by falcoctl)
Rules are published as plain files as well as OCI artifacts at each release

Check it out: https://github.com/falcosecurity/rules

Falcoctl is official 😎

The falcoctl project was promoted to "Official" status, and its repository is now part of the core.

What's Next? 🔮

It’s time to try out the new release! Here are some pointers for getting started with Falco:

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

The community is active on many things and there is no shortage of great ideas for next releases!

Thanks to all the people who wrote and tried plugins, we have great feedback for the next version. If you are a plugin developer or user, stay tuned for more APIs and functionality!

The modern eBPF probe is awesome and we want to keep improving it to get it out of the experimental stage 🚀

Falco maintainers also care a lot about the project’s own security. We’re exploring security-related considerations in the Falco Supply Chain Security Working Group. Join us if you can't wait to know more about this.

Stay Tuned 🤗

Join us in our communication channels and in our weekly community calls! It’s always great to have new members in the community and we’re looking forward to having your feedback and hearing your ideas.

You can find all the most up to date information at https://falco.org/community/.

Till the next release! 👋

Luca, Andrea, Teryl and Jacque

Blog: Falcosidekick 2.27.0 and Falcosidekick-UI 2.1.0

Tue, 10 Jan 2023 00:00:00 +0000

So many good things happened for Falcosidekick and Falcosidekick UI this year. It's still incredible these projects became so beloved and useful for the community. To all contributors, promotors and users, a big big thank you.

The new year is there, it's time to release new versions and reach 10 million Docker pulls for Falcosidekick.

Falcosidekick v2.27.0

What a huge release! Never has a previous release gotten so many new features and outputs. You can read the full changelog here.

New outputs

This release brings a lot of new outputs thanks to our amazing contributors. Here you have a list of them.

Yandex Data Stream

Yandex is a Russian cloud provider that provides various services such as Data Streams. With this new output, we can connect Falco to one more cloud providers. Thank you, preved911.

MQTT and Node-Red

IoT is a whole new world for Falco. With these 2 new outputs, Falco can make its first steps in this ecosystem and we are sure more will come in 2023. Stay tuned.

Zincsearch

Do you want a full-text indexer lighter than Elasticsearch? Take a look at Zincsearch.

Gotify

By using Gotify and the new dedicated output, you can now push Falco events to your Android phone.

Spyderbat

Are you a user of Spyderbat and want to extend its sources of events? Now you can thank spyder-kyle.

Tekton

Do you remember the blog post how to create a Response Engine for Falco with Tekton? The proposed solution used the generic Webhook output. From now on, Tekton can use a dedicated one.

TimescaleDB

TimescaleDB is an OSS database made for time-series data, thanks to jagretti Falcosidekick can insert into it the Falco events.

AWS Security Lake

At re:Invent 2023, AWS announced a new data lake made for security data: AWS Security Lake. We worked with AWS teams to have Falco as a source partner from day one, making it the first OSS project that can be used with that service and strengthening once more the integration with the AWS ecosystem.

New features

The list of new outputs is already quite long, but the list of enhancements is even longer. The full list is here, but let's have a look at the major changes.

SASL auth mechanisms for SMTP and Kafka outputs

Improving security is our duty to all, and one key element is the authentication method. Thanks to Lowaiz, both SMTP and Kafka outputs can now use the benefits of SASL Auth mechanisms.

Environment variables for custom labels and templated labels

The ability to inject custom fields in the payloads is an important feature of Falcosidekick. The only drawback was these fields were previously static. That limitation is over. Now, you can use environment variables in your custom fields. A new kind of custom fields has become available: `templated fields.' They allow the reuse of the present fields to generate new ones following with Go template:

templatedfields: # templated fields are added to falco events and metrics, it uses Go template + output_fields values
 Dkey: '{{ or (index . "k8s.ns.labels.foo") "bar" }}'

Hostname field

Since Falco 0.33, a new field is present in Falco events: hostname. Falcosidekick and all its current outputs are up to date and ready for it. Once again, thanks to Lowaiz.

Loki format and Grafana Cloud

The Loki format has been upgraded and credentials can be set. It allows you to use Grafana Cloud as a target.

K8S Policy Reports are binded to the namespaces

Policy Reports in K8S are still prototypes but Falcosidekick is already able to create them. Some improvements have been made to determine the target resource and the report is now created in the same namespace as the source pod.

More headers in SMTP payload

To avoid being flagged as spam by some anti-spam systems, some headers like From, To and Date have been added to the emails created by Falcosidekick.

CEF format Syslog

For the Syslog output, you can choose between json and CEF as formats. It makes easier the integration with some services like Microsoft Sentinel or Splunk.

Fixes

Even if we do our best to avoid them, the community has lately faced some bugs that we have fixed in this release.

The most important one was a race condition when headers were added to the POST requests. Adopters with high rates of requests were occasionally facing authentication failures or missing headers. bc-sb solved this with a temporary solution, but we'll improve it in the future (Falcosidekick v3? Who knows...).

Falcosidekick UI v2.1.0

The new features for Falcosidekick UI, although lower in number, are still big improvements. The full changelog is here.

Env vars for settings

All settings to configure Falcosidekick UI can be passed as either CLI arguments or as env vars. Run falcosidekick-ui --help for more details.

New logs

The logs were too verbose for production contexts. Now it's configurable via a log-level option:

-l string
 Log level: "debug", "info", "warning", "error" (default "info", environment "FALCOSIDEKICK_UI_LOGLEVEL")

Auto refresh

Long-term adopters may remember the dashboard in Falcosidekick UI v1 was auto-refreshed. This feature is back, for all widgets, independently of the page.

Authentication

This is a major new feature. The interface is now protected by the Basic Auth method. More methods will be added in the future:

Set the FALCOSIDEKICK_UI_USER env var to define the credentials.

Info page

The info page has been rewritten for a nicer look & feel.

Hostname

As for Falcosidekick, Falcosidekick UI supports the display of the new hostname field.

TTL for keys

Falcosidekick UI can store a huge amount of events, leading to filling the disk of the Redis database. A TTL for the entries can be set to avoid this situation.

-t int
 TTL for keys (default "0", environment "FALCOSIDEKICK_UI_TTL")

Conclusion

The respective Helm charts are already updated to allow you to test on your own all these great new features. Run a helm upgrade --reuse-values -n falco to do so.

Once again, thanks to all adopters and contributors who helped and contributed for years to create pieces of software useful to everybody. We hope 2023 will be amazing for Falco and its ecosystem.

As usual, if you have any feedback or need help, you can find us at any of the following locations.

Get started in Falco.org
Check out the Falcosidekick project on GitHub.
Check out the Falcosidekick UI project on GitHub.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Falco 0.33.1

Thu, 24 Nov 2022 00:00:00 +0000

Today we announce the release of Falco 0.33.1 🦅!

Novelties 🆕 and Fixes 🐛

Here's a tiny patch release! It only fixes two bugs reported by the community:

CrashLoopBackOff in some cases when the gVisor integration is enabled on Kubernetes (reported on Minikube and some versions of GKE)
Crash when the eBPF probe is used and one or more CPUs are switched off. Thanks FedeDP!

Thanks to everyone who reported and worked on issues!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.33.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

So many great things are happening in the Falco community right now. After meeting our friends at KubeCon NA, we're back at work with new features for the upcoming 0.34.0 release coming early 2023 with an unbelievable amount of work being done in the new eBPF probe, enhancements to falcoctl to make management of rules and plugins easier and much more!

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy 🎉

Luca

Blog: Falco 0.33.0 a.k.a. "the pumpkin release 🎃"

Wed, 19 Oct 2022 00:00:00 +0000

Dear community, today we are happy to announce the release of Falco 0.33.0 🎉!

A big thank you to the community for helping get the latest release over the finish line. The Falco community rallied behind this release and we wanted to share some of the latest novelties you’ll find in this most recent launch. To read a more detailed account of the release, check out v0.33.0 in the changelog.

What’s New? 🗞️

In this release we saw more than 160 pull requests across the repos of Falco and the libraries. We had a total of 20+ individual contributors. We’d like to give a special shout-out to Andrea Terzolo and Melissa Kilby for standing out as two of the most high-impact contributors for this release.

The project really seems to be more alive than ever! Thank you to our maintainers and contributors, as this would not happen without your support.

Updates - TLDR; 🚀

In release v0.33.0 the community focused on addressing the following updates & changes:

Libs now allow individual selection of which syscalls to collect during live captures, which helps Falco improve performance and reduce dropped events
Introduced the Kernel Crawler, a new tool that automatically identifies the most up to date kernel versions supported by popular distros
Syscall kernel ring-buffer size is now customizable for your environment needs
Mitigations for libsinsp’s Kubernetes metadata client to address recent issues that caused Falco to crash
Support for multiple simultaneous event sources, which means that you can now run multiple event sources in the same Falco instance
Added minikube as a supported platform in the driver loader and included it in our driver build matrix
Rule alert rate limiter is now optional and disabled at default
Support for two new syscalls and many improvements to the default Falco security ruleset

Selecting Interesting Syscalls ⚙️

A historical challenge when using Falco with a large system was to keep up with large amounts of kernel events. In the past, this was mitigated by what used to be called “simple consumer mode”, through which Falco discarded kernel events that were not useful for runtime security purposes. However, we lacked support for individually selecting which syscalls had to be collected and which to discard. This feature has been requested by the community for a while, as it is a great bonus point for both Falco and all other projects based on top of the Falco libraries. In this release, we refactored the whole system and introduced new libsinsp APIs that allow to individually select which syscalls and tracepoint events need to be instrumented for collection in the kernel. Now, Falco has higher control over collected security events, and is able to improve performance and reduce the amount of dropped events. At the same time, other projects can easily consume only the events they need without any additional instrumentation overhead.

Kernel Crawler 🔍

When deploying Falco, one of the biggest challenges has been to compile its drivers (kernel module or eBPF probe) for the specific kernel versions and customization you wish to instrument. To help our community, the Falco project has created prebuilt kernel modules and eBPF probes for widely-adopted distros and kernel versions. We have also provided a "driver loader" script that takes care of downloading and installing them before attempting local compilation. The build matrix has so far been constructed manually depending on the community demand and contributions, which makes it very hard to keep up with the most recent kernel versions.

Recently, the Kernel Crawler joined the Falco ecosystem as a tool that automatically searches for the most up to date kernels supported by multiple Linux distros (huge thanks to Federico Di Pierro for leading the effort). This helped us to dramatically expand our driver build matrix, and keeps it up to date with the latest kernel versions supported by the most popular distros without the need of manual intervention. This is a major step forward for Falco’s adoption, which we now expect to grow even further. Moreover, the Kernel Crawler populates an open database with all the information it collects. This is both a reference of the kernel versions and the distros supported by Falco, and a useful source of information for communities working in the space of kernel instrumentation like we couldn’t find on the internet so far.

Customizing the Syscall Kernel Ring-Buffer Size 💍

The ring-buffer is the shared piece of memory between Falco and the drivers in which all kernel events are pushed upon collection for Falco to consume them. When Falco is not able to keep up with the high throughput of events pushed, the buffer becomes full and some events are inevitably dropped.

Thanks to the great effort driven by Andrea Terzolo and Melissa Kilby, the syscall kernel ring-buffer size is now variable and configurable. In some cases, tuning this size may lead to better performance and less event drops on certain machines and environments. If you’re interested, check out the discussion at falcosecurity/libs#584.

Mitigations for Kubernetes Metadata Client ☸️

Starting from June’s Falco release, we included minor fixes for the Kubernetes client bundled inside libsinsp. This is the piece of code responsible for downloading metadata from your API server and populating fields in your security rules such as k8s.deployment.name, k8s.rc.name, etc. However, this causes Falco to receive too much data in certain situations, and to eventually crash. You can find more details in the following issue: falcosecurity/falco#1909.

Finding a stable and permanent solution is still being researched, as the problem of data overload has some intrinsic complexity. In this release, we introduced some short term solutions that prevent Falco from crashing in those scenarios by discarding useless information and handling errors gracefully. However, the big problem identified is that the Kubernetes cluster provides too much data, and we will keep looking for optimal solutions to this challenge in the future.

Running Multiple Simultaneous Event Sources 🚴

Wouldn’t it be nice if Falco could multi-task? Well, now it sorta can! We are delighted to announce that in this release Falco can now run multiple event sources in parallel. What does this mean? Well, it means that you can run plugins and syscall collections on the same Falco instance.

Historically, Falco supported consuming events from one source only. The only exception was the legacy support of the Kubernetes Audit Events, which allowed receiving those events and kernel events simultaneously. However, it was non-standard and has been substituted in favor of a plugin-based solution starting from Falco 0.32.0. Up until now, this meant that to consume events from more than one event source, users needed to deploy many instances of Falco, each configured with a different source.

This is a huge improvement and also brings back support for running syscall and k8s audit logs in the same Falco instance, for all the folks who were interested in doing so. For insights about the principles and rationale behind this release, follow the discussion at falcosecurity/falco#2074.

Please note that this feature introduces few user-facing changes to be aware of when updating. The primary one is that the syscall event sources will always be enabled by default if not explicitly disabled. So, please make sure you pass --disable-source=syscall to the Falco CLI if you’re interested in a plugin-only deployment! You can find more details in the documentation.

Supporting minikube in the Driver Loader 📥

We now offer new prebuilt drivers for the three most recent major version releases of minikube, which is a newly-supported platform for the Falco driver loader.

In general, it’s not possible to compile the Falco drivers locally when deploying on minikube, so in the past we needed to wait for a new minikube release to bundle the most recent Falco drivers. Thanks to the new Kernel Crawler, and great work carried out by Aldo Lacuku, our driver build grid now supports and auto-discovers the driver configurations for minikube and provides users with pre-built drivers to download with the driver loader. This reduces release delays to the bare minimum, and running Falco on minikube has never been easier!

Disabling Alert Rate Limiter at Default ❗

Falco provides a throttling mechanism for reducing the number of rule alerts, with the purpose of reducing noise in some environments. However, some users found concerns in this approach, as in the discussion at falcosecurity/falco#1333.

Falco v0.33.0 makes the rate limiter optional, and disables it in the default configuration, so that there is never a risk of discarding important alerts. At the same time, the feature is still present and configurable for everyone who needs to reduce Falco’s noise in their environment.

Updates on Syscall Coverage and Security Rules 🛡️

Call and you shall receive! Okay, that’s not exactly how that saying goes, but we acknowledged the importance of instrumentation coverage and critical updates to syscalls. After all, the power of Falco’s runtime security lies in the visibility it has over the system it gets deployed into. With this new release, Falco supports the collection of two new syscalls to ensure we keep those pesky hackers away: fsconfig and mlock2.

On top of that, there have been major updates to the default set of security rules bundled in Falco.

Since the last release, three new security rules have been added. Special thanks go to hi120ki for having been very active in maintaining the security rules over the past few months, and much of his work will be part of the next Falco releases as well. For v0.33.0, the new rules are:

Directory traversal monitored file read: detects attacks based on directory traversal
Modify Container Entrypoint: detects attacks based on CVE-2019-5736
Read environment variable from /proc files: detects attempts to read process environment variables

Additionally, existing rules have been updated to become less noisy and more optimized. Huge thanks to Melissa Kilby for taking the initiative to clean up the ruleset by disabling by default all the rules that were proved to never be triggered by Falco. This is a great step forward helping Falco be more performant by having fewer rules to evaluate at runtime.

What's Next? 🔮

It’s time to try out the new release! Here are some pointers for getting started with Falco:

Container Images
- falco (DockerHub, AWS ECR Gallery)
- falco-no-driver (DockerHub, AWS ECR Gallery)
- falco-driver-loader (DockerHub, AWS ECR Gallery)
CentOS/Amazon Linux
Debian/Ubuntu
openSUSE
Linux binary package

But the party is far from being over! The community is preparing lots of exciting updates for the near future. Special mention goes to the modern eBPF probe work led by Andrea Terzolo, which is under active development and should be rolled out by the next Falco release! Moreover, there has been plenty of work on falcoctl, and we can expect a new release of the tool to come soon and bring plenty of exciting novelties in the ecosystem!

Stay Tuned 🤗

You can find all the most up to date information at https://falco.org/community/.

See ya! 👋

Jason and Jacque

Blog: Falco 0.32.2

Tue, 09 Aug 2022 00:00:00 +0000

Today we announce the release of Falco 0.32.2 🦅!

Novelties 🆕

This release is really small, like a little 🐦, it only fixes the URL to download the falco BPF probe from Falco download page. A big thank you goes to eric-engberg, who proposed the fix, and as usual to everyone in the community for helping us in spotting these annoying bugs 🐛! You make Falco successful 🦅!

Thanks as always to the Falco maintainers for their support and effort during the entire release process.

Fixes 🐛

This release fixes just one bothersome bug:

The url from which Falco tryes to download the BPF probe was wrong, eric-engberg proposed the solution in this PR. Thank you again! 🙏

Try it! 🏎️

As usual, in case you just want to try out the stable Falco 0.32.2, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

It's an exciting time for Falco as we see so many great improvements and features. What's more exciting is the fact that there are many great ideas and awesome work going on to make the next big things happen.

Recently, there has been a lot of interest on the shiny new eBPF probe, making use of modern eBPF features like CO-RE, ringbuffer API and new tracing program.

In addition, many people in the community are interested in using Falco to read syscall events and plugin events simultaneously. If you are, I would suggest to take a look at the in-depth design for this new feature!

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Andrea

Blog: Falco 0.32.1

Mon, 11 Jul 2022 00:00:00 +0000

Today we announce the release of Falco 0.32.1 🦅!

Novelties 🆕

A bit more than a month has passed since the last release, and we already have 84 new commits in Falco (bringing the falcosecurity/falco repo to an even and eye pleasing total of 3,000 commits 😎) and a massive 215 commit changelog in libs!

A big THANK YOU 💖 goes as usual to everyone in the community for working on that many features and especially to those that are willing to test them even before release! You make Falco successful 🦅! Thanks as always to the Falco maintainers with their relentless work reviewing PRs and making sure the release process works smoothly.

Let's review some of the highlights of the new release.

What's new?

First of all, this new version comes with out of the box support for two technologies that were already partially or internally supported in previous versions and they're now getting the complete feature set they deserve, plus some minor CLI and configuration changes and under-the-hood improvements.

Official ARM AArch64 Packages and Images 🚀: you read it right! By popular demand, we now have official AArch64 support! It's hard to overstate the community effort that was required to make this happen, as the necessary changes span across multiple repos and touch pretty much all the components that make Falco work, including:

falcosecurity/falco with a ton of work by Federico Di Pierro to properly build Falco and create shiny new artifacts and images to make installation easy!
falcosecurity/libs thanks to Andrea Terzolo and Federico for adding kernel and eBPF support to this new architecture which, surprise surprise, behaves differently than x86_64 sometimes in really unexpected ways that are tricky to handle;
falcosecurity/test-infra and falcosecurity/driverkit thanks to David Windsor, Federico, maxgio92 and Michele Zuccala we now have proper infrastructure to build multi platform drivers and eBPF probes, now available in the download.falco.org repository!

gVisor support and relevant CLI options 🌕: gVisor is an application kernel for containers that provides efficient defense-in-depth anywhere. When using gVisor, in order to limit the attack surface of the host, each container is provided with its own application kernel. Normally, Falco would be unable to work in such environments since kernel modules or eBPF probes may not be installed within those sandboxes. Recently, the gVisor team developed a feature that allows security software to receive and audit syscalls that are executed by sandboxed containers; Falco can then use this stream of syscalls as a data source and monitor gVisor systems with the same rulesets as it normally would use. The relevant CLI options --gvisor-config, --gvisor-generate-config and --gvisor-root have been added for this purpose. Stay tuned for more information about how to make Falco and gVisor work together! I had the pleasure to work on this along with Lorenzo Susini. Thanks a lot to the gVisor team for their help!

If you are interested in the Falco libraries and drivers you will be happy to know that both libs and drivers are now versioned. In fact, this release uses libs version 0.7.0 and driver 2.0.0. Refer to the libs readme for more information about versioning strategies and release processes.

This release introduces some minor changes in the configuration, adding libs_logger.enabled and libs_logger.severity to be able to read libs logs which would otherwise be hidden from the user. The default behavior does not change but those options could be useful for troubleshooting and development.

Also, you can now see additional information about any configured plugin with the new --plugin-info CLI option.

Very worth mentioning is the big refactor that is going on in libscap (part of the falco libs) to make it easier to support different types of syscall sources. gVisor support leverages this feature, as the next big things most likely will.

New syscalls

The support for the dup family of syscalls has been enhanced, and also support for dup2 and dup3 is now available.

Fixes

Multiple bugs were fixed:

fixed incorrect behavior of the -V option when validating rules files;
fixed issues when loading kernel module with DKMS on Flatcar Linux and supporting fetching pre-built module/eBPF probe

...and much more of course, many of which are listed in the libs 0.7.0 release notes.

Not really a bug fix from previous releases but I'd like to add a shout out to Mauro Moltrasio for catching bugs early in reviews and also finding and fixing a tricky stability bug in the integration code for gVisor that was affecting the whole Falco, just right after I committed it!

Security Content 🔒

The default signature algorithm for the RPM package is now RSA/SHA256;
Bundled dependencies were upgraded, namely openssl to 1.1.1p and libcurl to 7.84.0.

Default rules update 🛡️

This release also includes updates to the default ruleset: 👇

Redirect STDOUT/STDIN to Network Connection in Container

Moreover, new rules were added: 👇

Java Process Class Download: detect potential log4shell exploitation

Try it!

As usual, in case you just want to try out the stable Falco 0.32.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

Recently, there has been a lot of interest on the shiny new eBPF probe, making use of modern eBPF features like CO-RE, ringbuffer API and new tracing program.

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Cheers 🎊

Luca

Blog: Falco 0.32.0

Fri, 03 Jun 2022 00:00:00 +0000

Today we announce the release of Falco 0.32.0 🦅!

Novelties 🆕

Let's review some of the highlights of the new release. This is one of the biggest releases ever, with around 200 commits on Falco and 230 on libs.

The Falco community once again proved to be super active, and we wanted to say a huge THANK YOU 🙏 💖 to everyone involved.

New features

This new release comes with a ton of inner rework; let's start with the foremost important change: Lua is no more a dependency of Falco! Ok, calm down now. Basically, the Falco rule loader was rewritten in C++, to achieve better performance. Moreover, the entire rule engine has been rewritten too.
This work reduces the workarounds in Falco, as it is now fully using libsinsp-provided filter parsers and compiler; finally, the new grammar fixes many minor and not-so-minor bugs.
Thanks to Jason Dellaluce for his amazing work!

Another effort by Jason was the porting of K8s Audit Log support to a plugin; consequently, there is no K8s Audit Log related code in Falco anymore.

Moreover, a new --list-syscall-events CLI option is now available, to print list of supported syscalls.

Users and groups management is now dynamic: newly added users/groups will be properly fetched by Falco. On host, their full informations will be retrieved; instead, on containers, only the uid and gid will be retrieved as there is no stable API to fetch user/group info. Moreover, Falco won't mix host and container users anymore.

Another big refactor happened on how Falco handles its CLI and config options, with the concept of "app actions". While this has no user facing changes, it is a big and noteworthy change.

Falco is now able to detect changes to ruleset or config file, and automatically restart itself. This behavior is enabled by default.

Two new operators were developed: bcontains and bstartswith. These are useful to perform byte matching on events raw data.
It allows better detection for log4shell like vulnerabilities.

Finally, all the Falco CI that is not involving any output artifact, has been ported to github actions. This frees up credits for CircleCI builds, mitigating various CI issues; moreover, it is now quicker.

New syscalls

As always, hard work was also spent on hardening the system, supporting new syscalls:

io_uring family of syscalls
mlock family of syscalls
capset syscall
open_by_handle_at syscall

Increased maturity of plugins

Plugins API reached stable 1.0.0, with tons of work to improve the API and its performance, eventually fixing any bug encountered. It means that the contract is now stable and you can start developing your own plugin; we eagerly wait for it!

GO Plugin Sdk was updated and all plugins were ported to new sdk.

Moreover, with this release, plugin related rules are shipped together with their plugin.

A couple of new plugins are now officially supported:

And more came from the community:

We are really pleased to see new plugins coming; hopefully Plugin API 1.0.0 will give it a boost!

Fixes

Multiple bugs were fixed:

a bug that caused Falco memory usage to skyrocket was solved. We are sorry for the inconvenience.
multiple issues with container events were fixed.
number of reported drops was mistakenly doubled while using the eBPF probe. This is now fixed.
multiple eBPF verifier issues were solved, resulting in a much more resilient probe.

...and much more!

Security Content 🔒

Bundled dependencies were upgraded, namely openssl to 1.1.1o and libcurl to 7.83.1, fixing a ton of CVEs!
Moreover, gRPC was also bumped to 1.44.0.

Rules update 🛡️

This release also includes updates to the default ruleset: 👇

Moreover, new rules were added: 👇

Try it!

As usual, to try out the stable Falco 0.32.0, you can install its packages following the process outlined in the docs:

Do you prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

Don't worry, we are still very hungry for improvements!

Current work is involving arm64 support (https://github.com/falcosecurity/falco/pull/1997, https://github.com/falcosecurity/falco/pull/1990, https://github.com/falcosecurity/driverkit/pull/143), gvisor event source support, and libs versioning with proper tags.

Moreover, a proposal for a shiny new eBPF probe was merged, and we anticipate the new probe will come to life very soon!
It will make use of modern eBPF features like CO-RE, ringbuffer API and new tracing program; together, these features will relax our support matrix while boosting performances.

Finally, maintainers are discussing about releasing a patch release (0.32.1), once arm64 support is complete.
We will keep you posted!

In the end, as always, the best is yet to come 😉

Let's meet 🤝

We meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy! 🥳

Federico

Blog: Falcosidekick 2.25.0 and Falcosidekick 2.0.0

Wed, 01 Jun 2022 00:00:00 +0000

A few days ago was the KubeCon EU in Valencia, Spain. The moment to meet contributors who made what Falcosidekick is now was a really enjoyable time and I hope we'll do it again in the future. One week before, two new major versions of Falcosidekick and Falcosidekick-Ui were released. Let's see what's new.

Falcosidekick

Almost 10 months without a new release for Falcosidekick, the version 2.25.0, and what a huge release is. For curious people, the full changelog can be found here.

New outputs

Each new release brings more outputs, thanks to the community. Here's the list of new ones:

Policy Report

With some CRD, you can now create reports in your Kubernetes clusters, the feature is often used for Security or Compliance, but anything is technically possible. For more details about how to use this output, read the documentation from @anushkamittal20 who implemented it for her project for Linux Foundation Mentorship Program.

apiVersion: wgpolicyk8s.io/v1alpha2
kind: ClusterPolicyReport
metadata:
 creationTimestamp: "2022-05-23T13:57:40Z"
 generation: 110
 name: falco-cluster-policy-report-4c9eac68
 resourceVersion: "71090"
 uid: ed8f0659-74d5-488c-90f8-d7b0622738cf
results:
- category: SI - System and Information Integrity
 message: Cluster Role Binding to cluster-admin role (user=%ka.user.name subject=%ka.req.binding.subjects)
 policy: Attach to cluster-admin Role
 properties:
 ka.req.binding: '%ka.req.binding'
 ka.user.name: '%ka.user.name'
 result: fail
 severity: high
 source: Falco
 timestamp:
 nanos: 98821031
 seconds: 40
- category: SI - System and Information Integrity
 message: Created Role/ClusterRole with write privileges (user=%ka.user.name role=%ka.target.name
 rules=%ka.req.role.rules)
 policy: ClusterRole With Write Privileges Created
 properties:
 ka.req.role: '%ka.req.role'
 ka.target.name: '%ka.target.name'
 ka.user.name: '%ka.user.name'
 result: fail
 severity: high
 source: Falco
 timestamp:
 nanos: 103148849
 seconds: 42

The reports can also be displayed with Policy Reporter UI, created by @fjogeleit another member of the Falco community.

Syslog

Years after its creation, Syslog remains a solid solution for managing the log files, especially if you're running Falcosidekick or else directly at the host level. With this new version, a Syslog server can be directly used as the target for the events, allowing you to send them in a secure place. Thanks to @bdluca.

AWS Kinesis

Do you want to ingest thousands of events from Falco and be able to run data analysis on them? You can do so smoothly with the new AWS Kinesis output, bringing a new integration of Falco with AWS Ecosystem. We would be delighted to know any use case with analytics the community could create now. Thanks to @gauravgahlot.

Zoho Cliq

Your DevOps/SRE/SecOps team uses Zoho Cliq for their communication? Allow them to receive nice notifications with this new output for Falcosidekick. Thanks to @averni.

Enhancements

Getting new features is nice, but we can also improve the existing ones, here's a list of major changes of this 2.25.0 release.

Compiled ans signed binaries

Until then, if you wanted binaries of Falcosidekick, you had to build them by yourself or use the provided Docker image. Now, each release will contain the compiled binaries for amd64 and arm64. The security is not forgotten, all artifacts are signed with Cosign.

Tags and Source

In January, Falco 0.31.0 brought its new Plugin system, the source field of events becoming more important. This new release of Falcosidekick updates all the outputs to deal with source and tags events. Your Response Engines can now be much clever than even.

IRSA

IRSA, aka Iam Role for Service Accounts, is the method provided by AWS for linking a Kubernetes Service Account with an IAM Role, allowing the Pod to easily use a Service. Falcosidekick is now able to use this mechanism for its outputs for AWS Services, no need to add Access and Secret Keys in your values.yaml. The UX is much better. Thanks to @VariableExp0rt.

Falcosidekick UI

I created the first version v0 of Falcosidekick-UI to have something more graphical for my talks, with the help of @fjogeleit we created a nice v1 that has been finally used more and more by people, becoming a famous product in the community. It was time to have a better version with some requested features:

a database (Redis) for a long term storage of events
an API for counting or searching the events
filters are kept as query strings, allowing to share links

All details to use this new version, v2.0.0, are described in the README.

Here's some screenshots:

Deployments

The Helm charts are already up to date, you can upgrade your deployments with:

helm upgrade falcosidekick falcosecurity/falcosidekick --set webui.enabled=true -n falco

helm upgrade falco falcosecurity/falco \
 --set falcosidekick.enabled=true \
 --set falcosdekick.webui.enabled=true -n falco

Enjoy

If you would like to find out more about Falco:

Get started in Falco.org.
Check out the Falco project in GitHub.
Get involved in the Falco community.
Meet the maintainers on the Falco Slack.
Follow @falco_org on Twitter.

Blog: Falco 0.31.1

Fri, 11 Mar 2022 00:00:00 +0000

Today we announce the release of Falco 0.31.1 🦅!

Novelties 🆕

Let's review some of the highlights of the new release.

New features

This release allows you to use multiple --cri command-line options (#1893) to specify multiple CRI socket paths. Note that Falco will only connect to the first one in order that successfully connects!

Speaking of command-line options, various changes are happening under the hood to improve the online help and to make it easier for contributors to add and modify options (#1886 #1903 #1915).

The update to the drivers version b7eb0dd brings in many improvements including proper detection of execveat, bugfixes for podman and support for the clone3 and copy_file_range system calls. In addition, the necessary extra arguments to entry system calls have been added to improve security of Falco event parsing as described below.

Security Content 🔒

Falco is now more resilient to TOCTOU type attacks that could lead to rule bypass (CVE-2022-26316). For more information, read the security advisory. Thanks to Xiaofei 'Rex' Guo and Junyuan Zeng for reporting this issue!

Default rules update

This release also includes modifications to the default ruleset, including a brand new rule to detect CVE-2021-4034 (Polkit Local Privilege Escalation) and false positive fixes (#1825, #1832)!

Try it!

As usual, in case you just want to try out the stable Falco 0.31.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

Falco 0.32.0 is anticipated to be released in May 2022!

As usual, the final release date will be discussed during the Falco Community Calls.

Let's meet 🤝

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors!

Enjoy! 🎉🔒

Luca

Blog: Falco 0.31.0 a.k.a. "the Gyrfalcon"

Mon, 31 Jan 2022 00:00:00 +0000

Today we announce the release of Falco 0.31.0, a.k.a the Gyrfalcon 🦅!

Gyrfalcons are the largest of the falcon species, just like this version of Falco has the biggest changelog ever released. To give you some metrics, since the last release, the falco and libs repositories counted 30+ individual contributors, 130+ pull requests, and 360+ commits 🤯. The Falco community proved to be more active than ever, and we wanted to say a huge THANK YOU 🙏 💖 to everyone involved.

The highlights

The changes are too many to list them all, so we'll just try to cover the highlights of the core features and topics. In case you want to go deep, here's the full Falco's changelog and the list of changes in libs.

Plugin system

Falco 0.31.0 finally ships with the brand new plugin system 🎉 ! Many things have changed since the initial proposal, and the feature is finally stable and production-ready.

Falco historically monitored system events from the kernel trying to detect malicious behavior on Linux nodes. In time, it got upgraded to also process K8S Audit Logs to detect suspicious activity in K8S clusters too. Now, the next step in the evolution of Falco is a plugin framework that standardizes how additional event sources can be attached to the engine and how more information can be extracted from those events.

Plugins can be written in almost any language of your preference. If you want to know more about how this works, take a look at the official documentation 📖. More or less, this is what the architecture of Falco looks like right now.

To do the honors, this release of Falco comes with the AWS Cloudtrail plugin and a new ad-hoc ruleset already packaged-in 📦 ! With these, Falco receives Cloudtrail logs from your infrastructure and sends alerts when suspicious activity happens, such as when the permissions of an S3 bucket are changed unexpectedly or when someone logs in without MFA. This is a great start to better integrating Falco into your infrastructure, and we expect more extensions like this to come!

Of course, you may wonder how hard it is to develop a Falco extension for your use cases. No worries, because the development experience was one of our top priorities, and we prepared two SDKs for writing Falco plugins in Go and C++:

Plugin SDK Go 👉 https://github.com/falcosecurity/plugin-sdk-go
Plugin SDK C++ 👉 https://github.com/falcosecurity/plugin-sdk-cpp

The SDKs are lightweight and allow you to develop Falco plugins with few lines of code! We put special attention to the Go SDK since Go is a well-appreciated language in the cloud-native community. Check out some examples and get started in a few minutes ⌚!

The Falco Community also maintains an official registry 📒 that keeps track of all the plugins acknowledged and recognized across the community. This serves both to make the plugin ecosystem more accessible to the community and for technical details such as reserving a specific plugin ID.

We expect plugins to be a game-changer, with the potential of making Falco evolve to the next level and become an all-in-one tool for cloud runtime security.

Drivers and libs improvements

Relevant performance optimization has been introduced in the drivers to drop all the non-monitored events right at the kernel level, which reduces ring buffer contention and decreases the drop rate 👉 libs#115.

The drivers added support to some new security-critical syscalls: openat2, execveat, mprotect! Also, the is_exe_writable flag was added to the execve syscalls family.

The eBPF probe received many improvements regarding stability and support for some compiler and kernel versions (e.g., with clang5, amznlinux2) 👉 libs#109, libs#140, libs#126, libs#96, libs#81, libs#179, libs#185.

Issues arising when processing huge container metadata have been solved by introducing a new LARGE block type, which dramatically increases the maximum block size supported 👉 libs#102.

Finally, a lot of effort has been put into upgrading critical dependencies and supporting more architectures and platforms 👉 libs#91, libs#164.

Other Falco novelties

Plugins apart, Falco received a few other significant updates:

Ability to set User-Agent HTTP header when sending HTTP output 👉 falco#1850.
Support to arbitrary-depth nested values in YAML configuration 👉 falco#1792.
Lua files used to load/compile rules are now bundled into the Falco executable 👉 falco#1843.
Linux packages are now signed with SHA256 👉 falco#1758.
Some fixes in the rule parser of the Falco engine 👉 falco#1777, falco#1775.
Finally, we moved the fully statically-linked build of Falco to another package, and the usual binary package switched back to a regular build (that was needed to allow plugins to be dynamically loaded). You can find both package flavors in our download repository.

Rule updates

The default ruleset 🛡️ includes few relevant new rules 👇

Existing rules, macros, and lists received some updates too, in particular with regards to possible bypasses 👇

What's next?

Many efforts are already ongoing to improve Falco's quality and stability. Two important proposals for libs (versioning and release process and API versioning for user/kernel boundary) are in the making. Meanwhile, the community is already thinking about a next-generation eBPF probe 🐝. Likely, many new plugins will come out soon 🚀 !

Furthermore, we believe it's time to renovate 🧹. For example, many parts of the codebase need to be re-designed or refactored: K8S Audit log should be rewritten as a plugin, various issues with the rule language parser/compiler, ARM compatibility should become officially supported, and much more.

So, stay tuned. The next release may surprise you 😉 !

Let's meet!

As always, we meet every week in our community calls. If you want to know the latest and the greatest you should join us there!

If you would like to find out more about Falco 👇

Get involved in the Falco community.
Check out the Falco project in GitHub.
Meet the maintainers on the Falco Slack.
Join the Falco mailing list
Follow @falco_org on Twitter.

Cheers 🥳 👋 !

Jason & Leonardo

Blog: Falco 0.30.0

Fri, 01 Oct 2021 00:00:00 +0000

Today we announce the fall release of Falco 0.30.0 🌱

This version includes new features, important fixes, and an exciting proposal for a libs plugin system!

Novelties 🆕

Let's review some of the highlights of the new release.

New features and fixes

This release introduces a new --k8s-node command-line option (#1671), which allows filtering by node name when requesting pod metadata to the K8s API server. Typically, it should be set to the node on which Falco runs. If empty, no filter is set, which may incur a performance penalty on large clusters. This new feature represents a significant performance improvement for Falco, and closes a long-waited fix to the issue confirmed by many deployments of Falco on production-scale Kubernetes clusters.

The update to the drivers version 3aa7a83 completes the performance enhancements for the collection of metadata from container orchestrators, and includes improvements to libsinsp public API, allowing consumers to modify the key parameters that determine the behavior of metadata collection from orchestrators like Kubernetes or Mesos. These parameters are now exposed as customizable settings in Falco, enabling users to tune metadata fetching behavior to their deployments. The default values are:

metadata_download:
 max_mb: 100
 chunk_wait_us: 1000
 watch_freq_sec: 1

This release also adds the ability to export rule tags and event source in gRPC and JSON outputs! This behavior can be configured, and enables Falco event consumers, such as Falco Sidekick, to take full advantage of Falco's event tagging feature. Happy tagging :)

Libs plugin system proposal

A proposal for a libs plugin system has been accepted, and we couldn't be more excited! The possibilities are limitless! 🎉

Plugins will allow users to easily extend the functionality of the libraries and, as a consequence, of Falco and any other tool based on the libraries. This proposal, in particular, focuses on two types of plugins: source plugins and extractor plugins. A source plugin implements a new sinsp/scap event source (e.g., "k8s_audit"), while an extractor plugin focuses on field extraction from events generated by other plugins, or by the core libraries.

Plugins are dynamic libraries (.so files in Unix, .dll files in windows) that export a minimum set of functions that the libraries will recognize. They can be written in any language, as long as they export the required functions. Go, however, is the preferred language to write plugins, followed by C/C++. To facilitate the development of plugins, a golang SDK has been developed.

Both the experimental plugin system and SDK are now incubating projects in the Falco organization, and include a set of initial examples. We invite the community to try them out, contribute new plugins, and join efforts to build together the foundation for cloud-native runtime security! 🚀

New Falco release schedule

Finally, after discussing with the community, a new release schedule has been approved for Falco. New releases are now due to happen three times per year: at the end of January, May, and September. We will continue to release hot fixes and minor patches in between major releases. As always, feedback, bug reports, and contributions are welcome! :)

Try it!

As usual, in case you just want to try out the stable Falco 0.30.0, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

You can also find the Falcosecurity container images on the public AWS ECR gallery:

What's next 🔮

Falco 0.31.0 is anticipated to be released in January 2022!

As usual, the final release date will be discussed during the Falco Community Calls.

Let's meet 🤝

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors! Falco reached 100 contributors, and all the other Falco projects are receiving a vital amount of contributions every day.

Special kudos to Falco Sidekick, which just passed the mark of 1.5M docker pulls on docker hub!

Keep up the good work!

Bye!

Fred

Blog: Falco 0.29.0

Mon, 21 Jun 2021 00:00:00 +0000

Today we announce the summer release of Falco 0.29.0 🌱

This version brings a lot of new features and fixes!

Novelties 🆕

Let's now review some of the new things Falco 0.29.0 brings.

New libraries repository!

As per this proposal - and as many of you probably already know - the repo falcosecurity/libs is the new home for libscap, libsinsp, and the Falco drivers.

With this release, also the last missing piece of the libs contribution is done: the building system is now updated to point to the new location and also the driver version is updated.

New libs version!

The update to the drivers version 17f5d brings new features/fixes:

support for tracing the userfaultd system calls
improvements to how libsinsp gathers Kubernetes pod resources limits and pod IP from the container runtime
improvement in libsinsp on pod metadata and namespace retrieval for large cluster scenarios, by getting them directly from container labels which is more efficient and use the K8s API server as a fallback
fixes to the issues reported by many users on Falco where you can't have a working BPF probe when compiling with Clang >= 10.0.0
fixes to correctly read, when loading the eBPF probe, the license from the BPF binary instead of always reading it from the libscap loader

Improvements on building system

Finally, it introduces necessary adaptations and improvements to make the Falco building system work with changes recently introduced in libs CMakefiles (in particular by PRs #23 and #30).

Updated rules

As usual, we keep improving the existing rules and we added new ones, like removing false positives when detecting non-sudo and non-root setuid calls.

Other false positives has been removed by ignoring additional known Kubernetes service account when watching for service accounts creted in kube-system namespace.

Improvements have been made also for anti-miner detection, by adding additional domains to be detected.

For a complete list please visit the changelog.

On the future

Now that the libscap, libsinsp, and the two Falco drivers have been contributed to the CNCF, we're moving in the direction of enabling people to benefit from those libraries by using them directly in their OSS projects, as now done by Falco.

For this reason we introduced a proposal (thanks to @leodido) about the versioning and the release process of the libs artifacts.

Try it!

As usual, in case you just want to try out the stable Falco 0.29.0, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

Notice that thanks to Jonah, one of our Falco Open Infra maintainers, you can find also the Falcosecurity container images on the public AWS ECR gallery:

This makes part of an effort to publish Falco container images on other registries that began while cooking up Falco 0.27.0.

Let's meet 🤝

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors! Falco reached 100 contributors, but also all the other Falco projects are receiving a vital amount of contributions every day.

Keep up the good work!

Ciao!

Max

Blog: Falco 0.28.1

Fri, 07 May 2021 00:00:00 +0000

Today we announce the spring release of Falco 0.28.1 🌱

This is our first patch release of Falco 0.28 that address some issues found.

And this release address some security advisories

You can take a look at the set of changes here:

0.28.1

As usual, in case you just want to try out the stable Falco 0.28.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

Notice that from this release onward, thanks to Jonah, one of our Falco Infra maintainers, you can find also the falco-no-driver container images on the AWS ECR gallery. Same for the the falco-driver-loader container images (link). This makes part of an effort to publish Falco container images on other registries that began while cooking up Falco 0.27.0.

Novelties 🆕

Let's now review some of the new things Falco 0.28.1 brings.

For a complete list please visit the changelog.

To highlitght some:

new flag --support it includes information about the Falco engine version.
new configuration field syscall_event_timeouts.max_consecutive to configure after how many consecutive timeouts without an event Falco must alert.
bug fix: don't stop the webserver for Kubernetes audit logs when some invalid data arrived.

Security Advisories

You can check all the security advisories in the page, but the ones important for this release are:

Let's meet 🤝

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors! Falco reached 100 contributors, but also all the other Falco projects are receiving a vital amount of contributions every day.

Keep up the good work!

Ciao!

Carlos

Blog: Falco 0.28.0 a.k.a. Falco 2021.04

Fri, 09 Apr 2021 00:00:00 +0000

Today we announce the spring release of Falco 0.28.0 🌱

This is the second release of Falco during 2021!

You can take a look at the set of changes here:

0.28.0

As usual, in case you just want to try out the stable Falco 0.28.0, you can install its packages following the process outlined in the docs:

Do you rather prefer using the container images? No problem at all! 🐳

You can read more about running Falco with Docker in the docs.

Novelties 🆕

Let's now review some of the new things Falco 0.28.0 brings.

For a complete list please visit the changelog.

Breaking changes

Before we dive into anything it's important to notice that this release introduces some breaking changes.

Since bintray is sunsetting 🌇, all the Falco packages, for all the officially supported distros, will be published at https://download.falco.org from now on.

We already moved the package repositories and the previous Falco versions (both development, starting from Falco 0.26.1 onward, and all the stable versions, starting with Falco 0.20.0).

So you can start using the new package repositories just now! Here's a step-by-step guide to upgrade your Falco repository settings. 📄

Do not use the Falco Bintray repositories anymore, please. ⚠️

Notice also that the DEB and RPM packages use now systemd ⚫◀️ in place of the previous init.d service units.

Another change worth mentioning is that we definitely removed the SKIP_MODULE_LOAD environment variable used by the Falco container image to skip the driver loading. It was deprecated with Falco 0.24.0. If you're still using is please switch to use the new environment variable named SKIP_DRIVER_LOADER. ⏭️

Exceptions

As announced, the support for structured rules exceptions has been merged in. ✔️

It's a mechanism to define additional conditions that when matched cause the Falco engine to do not emit the relative Falco alert.

You can read more about such a feature in the document proposing it.

Notice that the default Falco rulesets are not using the exceptions at the moment, but you can surely write your own Falco rules using this feature if it suits your needs.

Healthz

Thanks to Carlos, the Falco Kubernetes web server exposes now a /healthz endpoint.

It can be used to check whether Falco is up and running. It's a feature requested by the users of Falco Helm charts to improve them.

Falco driver loader

The Falco driver loader, a bunch of bash doing magic things when a Falco container starts, will first try to detect and download a prebuilt Falco driver for the current host (current list of prebuild drivers is available here), and only then it will try to compile a working Falco driver on the fly.

We decided to invert such logic because we have 4K+ prebuilt drivers and a mechanism to update them as soon new distro and new kernels born.

This way, the boot time of Falco containers should improve by a lot in the majority of cases, avoiding compiling a Falco driver for your host if we already built one for you.

Tunable drops

The syscall_event_drops configuration item inside falco.yaml gains a new child (threshold) that you can use to tune the noisiness of the drops.

It represents a percentage, thus you might provide a value between 0 and 1 for it. By default it's 0.1, feel free to experiment with it in case you need to.

Everything else

Engine fixes

A bug in the Falco engine, and precisely in the Falco rules language, preventing numbers to be parsed properly has been finally fixed.

Also, another bug regarding how the missing values (NA) were handled in multi-value fields (eg., lists) is now fixed and no more present.

Rules

As usual, our community is awesome at improving the Falco rules!

This release brings a bunch of improvements to various macros, lists, and rules. Take a look at the changelog (rules section) for details about them.

Three 3️⃣ new rules, Debugfs Launched in Privileged Container, and Mount Launched in Privileged Container, and Sudo Potential Privilege Escalation (very useful to promptly alert you about CVE-2021-3156) have also been introduced.

What's next 🔮

We have a scheduled 0.28.1 release on May 4th 2021!

As usual, the final release date will be discussed during the Falco Community Calls.

As always, we are going to have bug fixes and improvements.

Let's meet 🤝

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors! Falco reached 100 contributors, but also all the other Falco projects are receiving a vital amount of contributions every day.

Keep up the good work!

Bye!

Leo

Blog: Falco 0.27.0 a.k.a. "The happy 2021 release"

Mon, 18 Jan 2021 00:00:00 +0000

Today we announce the release of Falco 0.27.0 🥳

This is the first release of 2021!

You can take a look at the set of changes here:

0.27.0

As usual, in case you just want to try out the stable Falco 0.27.0, you can install its packages following the process outlined in the docs:

Do you rather prefer using the docker images? No problem!

You can read more about running Falco with Docker in the docs.

Important Falco 0.27.0 is the first release that has the container images released also on Amazon ECR. This is not officially supported yet and we are only releasing the falcosecurity/falco image there right now. Thanks to @leodido and @jonahjon!

What's new?

This is not a complete list, for a complete list visit the changelog.

Breaking changes

Before we dive into anything it's important to notice that this Falco release introduces one BREAKING CHANGE. If you rely on running Falco without any configuration file you can't do that anymore. All the official installation methods ships with a default configuration file with them.

Performance notes

The mechanism that handles Falco outputs has been completely rewritten in C++ (Thanks @leogr). Before this release, Falco relied on a mix of Lua and C++ API calls that led to a lot of crosstalk between the engine and the outputs mechanisms. Having a single C++ implementation helps a lot in reducing the crosstalk issue.

Since Lua is gone for the outputs now, the only reason that prevented us from having multi-threaded outputs is also gone. Outputs in Falco 0.27.0 are able to use multiple threads and also have a mechanism to detect when an output is too slow.

An output is "too slow" when it does not allow to deliver an alert within a given deadline, Falco will give an alert from the "internal" data source complaining about that. The default timeout is 200 milliseconds. It can be configured using the output_timeout configuration in falco.yaml.

Everything else!

New website As you can notice, we have a new website! Raji and Lore are the two behind this new restyle with the help of @leogr and @leodido. This new website features a new design, a search bar and a nice dropdown you can use to navigate old Falco releases (Falco 0.26 and 0.27 are the only ones available now).

gRPC changes The Falco gRPC version service now also exposes the Falco engine version.

Rules changes

We have 15 rules changes in this release! As always, our community values the quality of the rules as their top priority. Keeping a sane set of default rules for everyone to benefit is very important for us!

What's next?

We have a scheduled 0.28.0 release on March 18th 2021!

As always, we are going to have bug fixes and improvements.

A feature that is announced to go to 0.28.0 will be the support for structured rules exceptions, a way to define conditions to exclude certain alerts from happening when the exception happens.

You can read @mstemm's proposal here.

Moreover, we are very close to releasing ARM (armv7 and aarch64 builds) of Falco within the next releases. Lore worked on PR#1442 to port Falco to those architectures and Jonahjon is working to make our infrastructure support for building, testing and releasing for those as well.

Let's meet!

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Thanks to all the amazing contributors! Keep up the good vibes!

Bye!

Lore

Blog: Falcosidekick 2020

Tue, 12 Jan 2021 00:00:00 +0000

This fantastic post from @leodido about how has been the previous year 2020 for falco inspired me (link) I wanted to bring everyone up to speed on what we built for falcosidekick in 2020

Aside a lot of improvments and bug fixes, 8 new outputs have been integrated:

Rocketchat
Mattermost
Azure Event Hub
Discord
AWS SNS
GCP PubSub
Cloudwatch Logs
Apache Kafka

What really changed with previous releases was that almost all these outputs have been proposed and developed by other members of the falco community (kindly called the famiglia 😉). It warms my ♥️ a lot and makes me learn a lot about how to manage an open source project.

Thanks to everybody for your ideas, your comments, your help, your PR, your reviews, etc.

The following chart shows well how things are getting bigger and bigger for this small project that finally appeared useful for some people and companies.

A special 🙏 to @cpanato, @KeisukeYamashita and @nibalizer, who are now official maintainers of falcosidekick with me. 🎉 to them!

Last but not least, all my friendship to @danpopSD for his support and motivation. Merci mon ami.

What's next?

Release 2.20.0

Few times before this article is out we released one of the biggest versions since the beginning of falcosidekick. It results of a combination of a lot of efforts from many people.

The full changelog can be found here.

The main changes are three new outputs:

STAN (NATS Streaming)
PagerDuty
Kubeless (stay tuned, a post about this will be out soon 😉)

And ?

We believe the duo of falco + falcosidekick to be an obvious solution for most infrastructures, we are working hard to improve the code base and documentation. That will be all the major set of goals for the next major release 3.0.0 which is coming in the next few months. Until then with the help of n3wscott, we're working on adding the Cloudevents spec in a new HTTP output that will able to forward falco's events to more backends, like Knative.

Enjoy

Blog: Falco 0.26.2 a.k.a. "the download.falco.org release"

Tue, 10 Nov 2020 00:00:00 +0000

Today we announce the release of Falco 0.26.2 🥳

This one is a hotfix release for the Falco 0.26.1 released on October 1st.

You can take a look at the set of changes here:

0.26.2

As usual, in case you just want to try out the stable Falco 0.26.2, you can install its packages following the process outlined in the docs:

Do you rather prefer using the docker images? No problem!

You can read more about running Falco with Docker in the docs.

Why this release?

When you install Falco, you will either use a Kernel module, an eBPF probe or userspace instrumentation driver as described in the documentation.

As a service to our community, the Falco Infrastructure WG publishes pre-built drivers for all the current driver versions using the driverkit build grid.

Due to a spike in adoption in the month of October 2020, we had to come up with a better strategy for distributing our pre-built drivers.

To achieve this, we decided that from now on we will publish the drivers only to download.falco.org/driver instead of dl.bintray.com/falcosecurity/driver. Old drivers will be kept there to avoid disruption of current workloads but we will not publish new versions to the old bucket anymore. The PR that made this happen can be found here.

We also had a proposal that was discussed & approved for this change to happen, you can find it here

What should I do?

If you install Falco using a docker image and rely on our prebuilt drivers you have two options:

RECOMMENDED: Update to 0.26.2

Alternatively, you can change the DRIVERS_REPO environment variable in your current environment.

On bash:

export DRIVERS_REPO=https://download.falco.org/driver
falco-driver-loader

Docker

Pass it as environment variable using the docker run flag -e - for example:

docker run -e DRIVERS_REPO=https://download.falco.org/driver

Kubernetes

spec:
 containers:
 - env:
 - name: DRIVERS_REPO
 value: https://download.falco.org/driver

What's next?

We have a scheduled 0.27.0 release on December 1st!

It will contain a lot of exciting features and performance improvements! Stay tuned 🤙

Let's meet!

As always, we meet every week in our community calls, if you want to know the latest and the greatest you should join us there!

If you have any questions

Join the #falco channel on the Kubernetes Slack
Join the Falco mailing list

Bye!

Leo & Lore

Blog: Falco 0.26.1 a.k.a. "the static release"

Thu, 01 Oct 2020 00:00:00 +0000

Today we announce the release of Falco 0.26.1 🥳

This one is a hotfix release for the Falco 0.26.0 released last week!

You can take a look at the set of changes here:

As usual, in case you just want to try out the stable Falco 0.26.1, you can install its packages following the process outlined in the docs:

Do you rather prefer using the docker images? No problem!

You can read more about running Falco with Docker in the docs.

What's new?

From this Falco version onwards, if you download Falco using the tarball distribution (binary) or the falcosecurity/falco-no-driver container image, you will get a 100% statically-linked version of Falco! ⛓

The use case for this is that you can now download the tarball and copy the Falco binary (and configuration files) to any target machine or container without depending on the underlying system libraries, included libc.

The userspace working group is already using this in the experiments to bring Falco to new endeavors like AWS Fargate. 🕶

Rules

As always, our rules set is constantly improving and adapting to the constantly changing world. Many thanks to @ldegio, @mstemm, @csschwe and @leogr.

What's next?

As always, you have a chance to be part of this release by joining our community calls!

In the last Falco call the community chose to release Falco every 2 months, from now on. ⏰

So, we just created the milestone for 0.27.0 due by December 1st, 2020.

See you in winter!

Leo & Lore

Blog: Falco 0.25.0 a.k.a. "the summer release"

Tue, 25 Aug 2020 00:00:00 +0000

Today we announce the release of Falco 0.25 🥳

This one is a small release but a very important one!!

You can take a look at the set of changes here:

0.25.0

In case you just want to try out the stable Falco 0.25, you can install its packages following the usual process outlined in the docs:

Do you rather prefer using the docker images? No problem!

You can read more about running Falco with Docker in the docs.

What's new?

Driver changes

The driver now supports the renameat2 syscall and works with new kernel stable 5.8 releases!

Installation experience

Before, users had to install libyaml in their systems, it's not needed anymore.

Contributor experience

We have improved the contribution experience by rewriting the step by step instructions to run integration tests locally, the instructions can be found here.

Moreover, the build experience was improved as many users were reporting build problems on different operating systems, we took a chance to restructure our builds a bit and make them easier to work with.

Thanks to @leodido!

Outputs plugin developer experience

Outputs plugin developers and maintainers must be aware that gRPC was updated to 1.31.1. Please take your time to test and report issues, thank you!

Rules

As always, our rules set is constantly improving and adapting to the constantly changing world. Many thanks to @Kaizhe, @nvanheuverzwijn, @admiral0 and @leogr.

What's next?

We just created the milestone for 0.26.0 due by September 15, 2020. As always, you have a chance to be part of this release by joining our community calls!

See you soon!

Lore and Leo Grasso

Blog: Falco 0.24.0 a.k.a. "the huge release"

Thu, 16 Jul 2020 00:00:00 +0000

After two long months, look who's back!

Today we announce the release of Falco 0.24 🥳

You can take a look at the huge set of changes here:

0.24.0

In case you just want to try out the stable Falco 0.24, you can install its packages following the usual process outlined in the docs:

Do you rather prefer using the docker images? No problem!

You can read more about running Falco with Docker in the docs.

Breaking Changes

In case you wanna grab statistics about your running Falco instance, be aware that this PR fixed and changed the name of the CLI flag you need to enable such feature. The flag is --stats-interval now and finally, it also works for values greater than 999 milliseconds.

Because of performance issues of the Falco gRPC Outputs API we went through an almost complete redesign of the gRPC server and the outputs RPCs.

Long story short: the gRPC outputs method is now falco.outputs.service/get and not falco.outputs.service/subscribe anymore.

Furthermore, we introduced a falco.outputs.service/sub gRPC method that behaves in the same way the old one was behaving, except that it is way faster than the old method.

Notorious gRPC fixes and features

Some months ago, a user reported a very high CPU usage when using Falco gRPC outputs API with Falco 0.21.

Profiling the code we discovered that the gRPC threads were keeping the CPUs very very busy.

Digging deeply into the gRPC code and the gRPC core, Leo and Lore soon realized that to solve the issue a rewrite of important pieces of the Falco gRPC code was needed.

So we introduced a bidirectional API (falco.outputs.service/sub) to watch the Falco alerts through gRPC and we changed the server streaming gRPC outputs method (falco.outputs.service/get) to consume less memory and fewer CPU resources.

After some days of fine-tuning and continuous tests (4MLN requests towards the gRPC server, in 10 seconds) we've been able to reduce the CPUs occupancy of the gRPC outputs methods from nearly ~90% to values less than 20%. 🚀

In that PR you can find all the story, all the code changes, and also the instructions to quickly try out the new Falco gRPC output methods using grpcurl.

So, all's well that ends well: users are now happy and we too! 🤗

Finally, now that Falco gRPC outputs are better, we want to advertise the community about two other important and gRPC related features that Falco 0.24 ships:

you can now let Falco automatically configure the threadiness of its gRPC server by using threadiness: 0 into the Falco config (falco#1271)
lo and behold, you can now connect to the Falco gRPC server through a Unix socket (falco#1217)

We already updated the Falco Go client.

So, we'd invite all the Falco community and users to try out these new features and the improvements about gRPC!

Support for eBPF driver on CentOS 8 is back!

Since April some friends of our community reported issues on building the Falco eBPF driver on CentOS 8 (falco#1129).

After some intensive debugging sessions, Lorenzo and Leo discovered the cause: CentOS 8 backported process type functionalities (and relates structs) from Linux kernel 4.19 to 4.18 that made the driver checks ineffective.

Do you wanna look at some eBPF? Take a look at this PR!

Falco driver version 85c8895 contains the fix so that y'all can again run our beloved tool on your CentOS 8 boxes. 📦

Unbuffered outputs 😆

Leonardo Grasso finally spotted a tricky typo that was causing buffered_output: false config option to do not work as expected.

Thanks to his fix, from now on Falco will promptly output its alerts on stdout when this option is disabled.

Also, we'd like to welcome Grasso in the family of Falco maintainers!

Rules update

We are very thankful to Khaize for this huge PR that introduces a bunch of placeholder macros.

Thanks to his effort, users can now customize their own Falco rulesets more easily!

Some statistics

38 pull requests merged in, 29 of which containing changes directly targeting our end-users.

105 commits since past release, that was two months ago.

Be aware: userspace instrumentation is coming...

In this release Falco introduces userspace level instrumentation contract.

This functionality can be enabled by passing the -u flag when starting Falco, or using its long version - ie., --userspace.

A userspace implementation will also need to be implemented as well to take advantage of this contract.

The Falco community is currently working on an implementation called pdig which is built around ptrace(2) and seccomp. We are very excited to see pdig reach production support in the future.

Blog: Falco 0.23.0 a.k.a. "the artifacts scope release"

Mon, 18 May 2020 00:00:00 +0000

Another month has passed and Falco continues to grow!

Today we announce the release of Falco 0.23 🥳

Wondering why this release is called "The Artifacts Scope" release? Please read more here.

You can take a look at the whole set of changes here:

0.23.0

In case you just want to try out the stable Falco 0.23, you can install its packages following the usual process outlined in the docs:

Do you rather prefer using the docker images? No problem!

docker pull falcosecurity/falco-no-driver:latest # The most recent version
docker pull falcosecurity/falco-no-driver:0.23.0 # A specific version of Falco such as 0.23.0
docker pull falcosecurity/falco-driver-loader:latest # The most recent version of falco-driver-loader with the building toolchain
docker pull falcosecurity/falco-driver-loader:0.23.0 # A specific version of falco-driver-loader such as 0.23.0 with the building toolchain
docker pull falcosecurity/falco:latest # The most recent version with the falco-driver-loader included
docker pull falcosecurity/falco:0.23.0 # A specific version of Falco such as 0.23.0 with falco-driver-loader included

Please be aware that: we now recommend that instead of using falcosecurity/falco:latest directly, you use the falcosecurity/falco-driver-loader image first, then use the falcosecurity/falco-no-driver:latest. The falcosecurity/falco:latest is going nowhere, we just want to provide a way to do the same thing but splitted into two separate processes to lower the attack surface of the running Falco container. Read more about the images reorganization here.

Breaking Changes

Both in the packages and in the falco-driver-loader script now the kernel module and eBPF probe are referenced as falco.ko and falco.o respectively, before they were falco-probe.ko and falco-probe.o. In the case of Falco installed using the kernel module this can lead to a duplicated module loaded given that the names are different. Make sure you don't have a duplicated module by
The falco-driver-loader script environment variable to use a custom repository to download drivers now uses the DRIVERS_REPO environment variable instead of DRIVER_LOOKUP_URL. This variable must contain the parent URI containing the following directory structure /$driver_version$/falco_$target$_$kernelrelease$_$kernelversion$.[ko|o].

Rules update (yay yay! We always improve the default ruleset!!)

rule(Redirect STDOUT/STDIN to Network Connection in Container): correct rule name as per rules naming convention
rule(Redirect STDOUT/STDIN to Network Connection in Container): new rule to detect Redirect stdout/stdin to network connection in container
rule(K8s Secret Created): new rule to track the creation of Kubernetes secrets (excluding kube-system and service account secrets)
rule(K8s Secret Deleted): new rule to track the deletion of Kubernetes secrets (excluding kube-system and service account secrets)

Some statistics

35 pull requests merged in, 18 of which containing changes directly targeting our end-users.

72 commits since past release, that was a month ago.

Upcoming things

We are about to merge support for unix sockets in the Falco gRPC API with #1217, more importantly during this release cycle the community made a decision about adopting pdig as a repository (learn here what this means). pdig will allow Falco to run completely in userspace. This is very useful when Falco is deployed in environments where it's not possible to load a kernel module or an eBPF probe. Our community members, already created two projects that can be used to deploy Falco with pdig as a driver, falco-trace and falco-inject. We will look forward to adopting them or making different decisions.

See you next month with many more fabulous things!

Blog: Falco 0.22 a.k.a. "the hard fixes release"

Fri, 17 Apr 2020 00:00:00 +0000

Another month has passed and Falco continues to grow!

Today we announce the release of Falco 0.22 🥳

You can take a look at the whole set of changes here:

0.22.0 - thanks to Leonardo Grasso for his first ever release!
0.22.1 - hotfix by me and Lorenzo Fontana

In case you just want to try out the stable Falco 0.22, you can install its packages following the usual process outlined in the docs:

Do you rather prefer using the docker images? No problem!

docker pull falcosecurity/falco:0.22.1
docker pull falcosecurity/falco:0.22.1-minimal
docker pull falcosecurity/falco:0.22.1-slim

Notable Changes

This release comes with a lot of fixes for longstanding tough bugs!

But also with some new features 😊

In case you are managing the Falco drivers yourself please make sure to update them to version a259b4bf49c3330d9ad6c3eed9eb1a31954259a6 (reference here).

eBPF driver

Some users reported a problem in getting the eBPF driver to work on GKE.

This release finally introduces a fix for it.

values

Some users reported they were getting <NA> values for docker and Kubernetes metadata in the alerts.

With the following pull requests, falco#1133, falco#1138, and falco#1140, the problem should now be definitely fixed, as reported by the users testing the development release of Falco containing the fixes.

Falco version and driver version are now distinct!

Going through the process of a better modularization for Falco, now the Falco version and the version of its drivers are two distinct things finally!

Clearly, in order to obtain this some PRs were needed 😝 both in the packagin system and in the falco-driver-loader script.

Rules, rules everywhere!

This release also had a lot of rule changes. Most notably vicenteherrera created many new rules:

Full K8s Administrative Access
Ingress Object without TLS Certificate Created
Untrusted Node Successfully Joined the Cluster
Untrusted Node Unsuccessfully Tried to Join the Cluster
Network Connection outside Local Subnet
Outbound or Inbound Traffic not to Authorized Server Process and Port

Thanks Vicente! 🙌🏻

Synchronous CRI metadata fetch

Thanks to PR falco#1099 users can now disable the asynchronous fetching of CRI metadata forcing it to be synchronous.

To do it, just pass the --disable-cri-async flag to Falco.

While this can slow down Falco event processing and can cause drop rate to raise, it helps in getting less null values for containers metadata.

Before using this flag, please try out this release since it contains other fixes for this topic!

Some statistics

23 pull requests merged in, 18 of which containing changes directly targeting our end-users.

49 commits since past release, in 30 days.

Upcoming things

The drivers build grid is almost ready.

Just some missing refinements and then Falco will have again a set of prebuilt drivers (both kernel modules and eBPF probes) to be downloaded during the installation!

Blog: Falco 0.21.0 is out!

Wed, 18 Mar 2020 00:00:00 +0000

Even though there's the lockdown, Falco 0.21.0 decided to go out! Such a bad guy!

Notably, this is the first release that happens with the new build & release process. 🚀

In case you just want Falco 0.21.0, you can find its packages at the following repositories:

Instructions to install using them are already updated on the Falco website:

Instead, for people preferring docker images... 🐳

docker pull falcosecurity/falco:0.21.0
docker pull falcosecurity/falco:0.21.0-minimal
docker pull falcosecurity/falco:0.21.0-slim

Notable Changes

Release #100 of Falco comes with some notable changes.

New release process in place

During past weeks, I worked together with Lorenzo to put in place a completely new and automated release process for Falco.

We did most of the work into PR 1059.

This process takes place in two cases:

A pull request is merged into master, which leads to the release of a development version of Falco
A commit on master receives a git tag, which leads to the release of a stable version of Falco

When one of these two conditions happen:

it packages Falco into signed (GPG public key) packages: DEB, a RPM, and a TAR.GZ
it pushes these packages to their new open repositories
1. deb-dev, rpm-dev, bin-dev for development versions
2. deb, rpm, bin for stable versions
it builds the docker images from these packages
it pushes the docker images to the docker hub
1. falcosecurity/falco:master, falcosecurity/falco:master-slim, falcosecurity/falco:master-minimal for development versions
2. falcosecurity/falco:latest, falcosecurity/falco:latest-slim, falcosecurity/falco:latest-minimal for stable versions

March 2021 update: All packages are now published to download.falco.org.

FALCO_BPF_PROBE

Thanks to Lorenzo contribution (PR 1050), to make Falco use the eBPF probe as a driver you need to specify an environment variable named FALCO_BPF_PROBE, not SYSDIG_BPF_PROBE anymore.

FALCO_BPF_PROBE="" ./build/release/userspace/falco/falco -r ...

Please update your systemd scripts or Kubernetes deployments.

Falco versions are now SemVer 2.0 compliant

In PR 1086, I completed the process of creating the Falco version as SemVer 2.0 compliant version strings, from the git index.

This PR introduces the pre-release part into Falco versions.

Now Falco versions are something like 0.21.0-3+c5674c9, where 3 is the number of commits since the latest stable version (0.21.0) of Falco, while c5674c9 is the commit hash of the current development version.

Please notice that the Falco gRPC version API already contains this version part, too.

Detect outbound connections to common miner pool ports rule disabled by default

Thanks to Khaize work in PR 1061 users will not be hit from a tedious amount of alerts about hypothetical mining tools.

From now on, this rule is disabled by default.

Also, if it is enabled by you, it will ignore localhost and RFC1918 addresses.

Other changes

You can read the full changelog here!

Some statistics

19 PRs merged in, 12 of which containing changes targeting end-users.

64 commits since past release, in 17 days.

Upcoming things

Stay tuned for the upcoming drivers build grid which, using driverkit - a quarantine project by me and Lorenzo - will pre-build and release (in the open too!) the Falco kernel modules and the Falco eBPF probes for a set of predefined target systems and kernel releases.

Falco – Release

Blog: Introducing Falco Operator 0.2.0

What's new? TL;DR

The road to production readiness

Major features and improvements

Ecosystem components

ConfigMap support for rules and configuration

Structured API types

Redesigned OCI artifact API

Reference tracking with finalizers

Enhanced observability

Update strategy support

Server-Side Apply

Breaking changes ⚠️

Summary of breaking changes

A Helm chart is on its way

Meet us at KubeCon

Try it out

Full stack quickstart

Step by step

Stay connected

Blog: Introducing Falco 0.43.0

What's new? TL;DR

Latest updates

Deprecations

Legacy eBPF probe deprecation

gVisor deprecation

gRPC output and server deprecation

GPG key rotation

Container plugin improvements

Falcoctl tweaks and improvements

follow polling interval increase to 1 week

Dependency resolution improvements

Support for cosign v3

Key fixes

evt.arg.filename field reintroduction

Falcoctl signature verification fixes

Signature verification fix for full reference artifacts

Signature verification fix for authenticated registries

Breaking changes and deprecations

Bump drivers minimum required kernel version to 3.10

Deprecation warnings

Try it out

Stay connected

Blog: Introducing Falco 0.42.0

What's new? TL;DR

Major features and improvements

Capture recording feature

Drop enter initiative

Plugin event schema versioning

Thread table auto-purging configuration

Static fields

Breaking changes and deprecations ⚠️

Event direction and evt.dir deprecation

Plugin API changes

Deprecation warnings

Try it out

Stay connected

Blog: Introducing Falco 0.41.0

What’s new? TL;DR

Major features and improvements

Reimplemented container engines support

Kubernetes operator

Security best practices improvements

Breaking changes and deprecations ⚠️

Removed command line options and equivalent configuration options

Behavior changes

Dropped features

Deprecations

Try it out

What’s next?

Stay connected

Blog: Falco Talon v0.3.0

What's new?

Try it! 🏎️

Let's meet 🤝

Blog: Falcosidekick 2.31.0

New output

OTLP Metrics

New features

`follow` polling interval increase to 1 week

`evt.arg.filename` field reintroduction

Bump drivers minimum required kernel version to `3.10`

Event direction and `evt.dir` deprecation