Docs: Introduction to Feast

Mon, 01 Jan 0001 00:00:00 +0000

Alpha

This Kubeflow component has alpha status with limited support. See the Kubeflow versioning policies. The Kubeflow team is interested in your feedback about the usability of the feature.

Use Feast for defining, managing, discovering, validating, and serving features to your models during training and inference.

This page introduces feature store concepts as well as Feast as a component of Kubeflow.

Introduction to feature stores

Feature stores are systems that help to address some of the key challenges that ML teams face when productionizing features

Feature sharing and reuse: Engineering features is one of the most time consuming activities in building an end-to-end ML system, yet many teams continue to develop features in silos. This leads to a high amount of re-development and duplication of work across teams and projects.
Serving features at scale: Models need data that can come from a variety of sources, including event streams, data lakes, warehouses, or notebooks. ML teams need to be able to store and serve all these data sources to their models in a performant and reliable way. The challenge is scalably producing massive datasets of features for model training, and providing access to real-time feature data at low latency and high throughput in serving.
Consistency between training and serving: The separation between data scientists and engineering teams often lead to the re-development of feature transformations when moving from training to online serving. Inconsistencies that arise due to discrepancies between training and serving implementations frequently leads to a drop in model performance in production.
Point-in-time correctness: General purpose data systems are not built with ML use cases in mind and by extension don’t provide point-in-time correct lookups of feature data. Without a point-in-time correct view of data, models are trained on datasets that are not representative of what is found in production, leading to a drop in accuracy.
Data quality and validation: Features are business critical inputs to ML systems. Teams need to be confident in the quality of data that is served in production and need to be able to react when there is any drift in the underlying data.

Feast as a feature store

Feast is an open-source feature store that helps teams operate ML systems at scale by allowing them to define, manage, validate, and serve features to models in production.

Feast provides the following functionality:

Load streaming and batch data: Feast is built to be able to ingest data from a variety of bounded or unbounded sources. Feast allows users to ingest data from streams, object stores, databases, or notebooks. Data that is ingested into Feast is persisted in both online store and historical stores, which in turn is used for the creation of training datasets and serving features to online systems.
Standardized definitions: Feast becomes the single source of truth for all feature definitions and data within an organization. Teams are able to capture documentation, metadata, and metrics about features. This allows teams to communicate clearly about features, test feature data, and determine if a feature is both safe and relevant to their use cases.
Historical serving: Features that are persisted in Feast can be retrieved through its feature serving APIs to produce training datasets. Feast is able to produce massive training datasets that are agnostics of the data source that was used to ingest the data originally. Feast is also able to ensure point-in-time correctness when joining these data sources, which in turn ensures the quality and consistency of features reaching models.
Online serving: Feast exposes low latency serving APIs for all data that has been ingested into the system. This allows all production ML systems to use Feast as the primary data source when looking up real-time features.
Consistency between training and serving: Feast provides a consistent view of feature data through the use of a unified ingestion layer, unified serving API and canonical feature references. By building ML systems on feature references, teams abstract away the underlying data infrastructure and make it possible to safely move models between training and serving without a drop in data consistency.
Feature sharing and reuse: Feast provides a discovery and metadata API that allows teams to track, share, and reuse features across projects. Feast also decouples the process of creating features from the process of consumption, meaning teams that start new projects can begin by simply consuming features that already exist in the store, instead of starting from scratch.
Statistics and validation: Feast allows for the generation of statistics based on features within the systems. Feast has compatibility with TFDV, meaning statistics that are generated by Feast can be validated using TFDV. Feast also allows teams to capture TFDV schemas as part of feature definitions, allowing domain experts to define data properties that can be used for validating these features in other production settings like training, ingestion, or serving.

Next steps

Please follow the Getting Started with Feast guide to set up Feast and run walk through our tutorials.

Resources

Docs: Getting started with Feast

Mon, 01 Jan 0001 00:00:00 +0000

This guide provides the necessary resources to install Feast alongside Kubeflow, describes the usage of Feast with Kubeflow components, and provides examples that users can follow to test their setup.

For an overview of Feast, please read Introduction to Feast.

Installing Feast with Kubeflow

This guide assumes that you have a running Kubeflow cluster already. If you don’t have Kubeflow installed, then head on over to the Kubeflow installation guide.

Feast can be installed into a running Kubeflow deployment. The process of installing Feast into Kubernetes is documented in the Deploying Feast into Kubernetes guide. Please follow this guide to install Feast into the same Kubernetes cluster as Kubeflow.

Accessing Feast from Kubeflow

Once Feast is installed within the same Kubernetes cluster as Kubeflow, users can access its APIs directly without any additional steps.

Feast APIs can roughly be grouped into the following sections:

Feature definition and management: Feast provides both a Python SDK and CLI for interacting with Feast Core. Feast Core allows users to define and register features and entities and their associated metadata and schemas. The Python SDK is typically used from within a Jupyter notebook by end users to administer Feast, but ML teams may opt to version control feature specifications in order to follow a GitOps based approach.
Model training: The Feast Python SDK can be used to trigger the creation of training datasets and generation of statistics through Feast Serving. The most natural place to use this SDK is to create a training dataset as part of a Kubeflow Pipeline prior to model training.
Model serving: Feast provides three different SDKs for online feature serving, a Python SDK, Java SDK, and Go SDK. These clients are used prior to inference with Model Serving systems like KFServing, TFX, or Seldon.

All of the above clients interact with Feast through gRPC endpoints (Core, Serving). These APIs allow users to directly interface with Feast services if they do not wish to use an SDK.

Examples

Feast also comes with example notebooks that users can use to get up to speed.

Next steps

For more details on Feast concepts please see the Feast documentation
Please see our changelog and roadmap for new or upcoming functionality.
Please use GitHub issues for any feedback, issues, or feature requests.
If you would like to get involved with Feast, come and visit us in #Feast in the Kubeflow Slack, or join our community calls, mailing list, or have a look at our contribution process

Kubeflow – Feature Store

Docs: Introduction to Feast

Alpha

Introduction to feature stores

Feast as a feature store

Next steps

Resources

Docs: Getting started with Feast

Installing Feast with Kubeflow

Accessing Feast from Kubeflow

Examples

Next steps