polaris

Apache Polaris

Apache Polaris™ is an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg’s REST API, enabling seamless multi-engine interoperability across a wide range of platforms, including Apache Doris™, Apache Flink®, Apache Spark™, Dremio® OSS, StarRocks, and Trino.

Documentation is available at https://polaris.apache.org. The REST OpenAPI specifications can be browsed online: Polaris Management API and Polaris Catalog API.

For a high-level, auto-generated tour of the codebase’s modules and relationships, see the Code Wiki for Apache Polaris. It is a third-party, auto-generated navigation aid, useful for orientation, but the source tree remains the authoritative reference.

To get involved, subscribe to the dev mailing list (or browse the archives) and read the CONTRIBUTING guide.

Overview & Quickstart

• Overview of what Polaris is and how it works.
• Quickstart to set up a Polaris instance locally or on a supported cloud provider.

Project Structure

The repository is organized into the following groups of Gradle modules. Each leaf is a Gradle subproject; some link to a per-module README.

• Core
  • polaris-core - entity definitions and core business logic
• API - generated from the OpenAPI specifications:
  • polaris-api-management-model, polaris-api-management-service - Polaris Management API
  • polaris-api-iceberg-service - Iceberg REST service
  • polaris-api-catalog-service - Polaris Catalog API
• Runtime
  • polaris-server - Quarkus-based server
  • polaris-admin - admin tool, mainly for bootstrapping persistence
  • polaris-runtime-service - the Polaris service package
  • polaris-runtime-defaults - default runtime configuration
  • polaris-distribution - distribution packaging
  • polaris-runtime-common, polaris-runtime-test-common - shared runtime and test utilities
  • polaris-runtime-spark-tests - integration tests for the Spark plugin
• Persistence
  • polaris-relational-jdbc - JDBC implementation of BasePersistence
• Extensions
  • polaris-extensions-federation-hive, polaris-extensions-federation-hadoop, polaris-extensions-federation-bigquery - catalog federation
  • polaris-extensions-auth-opa, polaris-extensions-auth-ranger (plus *-tests) - external authorization
• Tooling & build support
  • polaris-bom, polaris-build-logic, polaris-version - BOM, shared build logic, versioning
  • polaris-immutables, polaris-misc-types, polaris-container-spec-helper - shared utilities
  • polaris-floci-aws-testcontainer, polaris-floci-az-testcontainer, polaris-floci-gcp-testcontainer, polaris-rustfs-testcontainer, polaris-hms-testcontainer - test containers
  • polaris-config-docs-{annotations,generator,site} - reference-doc generation
  • aggregated-license-report - aggregated license report
• Tests
  • polaris-tests - normative integration tests, reusable downstream

Other top-level directories:

• spec/ - OpenAPI specifications
• client/python/ - Python client
• plugins/spark/ - Polaris Spark plugin
• regtests/ - regression tests
• helm/ - Helm charts
• site/ - documentation site
• codestyle/, gradle/, server-templates/ - build, style, and codegen support

Additional tooling lives in the separate Polaris-Tools repository.

Building and Running

Apache Polaris is built using Gradle with Java 21+ and Docker 27+.

• ./gradlew build - To build and run tests. Make sure Docker is running, as the integration tests depend on it.
• ./gradlew assemble - To skip tests.
• ./gradlew check - To run all checks, including unit tests and integration tests. To run all checks except integration tests, use ./gradlew check -PnoIntegrationTests.
• ./gradlew run - To run the Polaris server locally; the server is reachable at localhost:8181. This is also suitable for running regression tests, or for connecting with Spark. Set your own credentials by specifying system property ./gradlew run -Dpolaris.bootstrap.credentials=POLARIS,root,secret where:
  • POLARIS is the realm
  • root is the CLIENT_ID
  • secret is the CLIENT_SECRET
  • If credentials are not set, it will use preset credentials POLARIS,root,s3cr3t
• ./regtests/run_spark_sql.sh - To connect from Spark SQL. Here are some example commands to run in the Spark SQL shell:

create database db1;
show databases;
create table db1.table1 (id int, name string);
insert into db1.table1 values (1, 'a');
select * from db1.table1;

• env POLARIS_HOST=localhost ./regtests/run.sh - To run regression tests locally, see more options here.

Makefile Convenience Commands

• make build-server, make build-admin build components and container images
• make minikube-start-cluster, make minikube-cleanup manage a local minikube cluster
• make helm-doc-generate, make helm-unittest work with the Helm charts
• make client-lint, make client-regenerate work with the Python client
• make install-dependencies-brew install developer prerequisites on macOS

Running in Docker

Build the image locally:

./gradlew \
  :polaris-server:assemble \
  :polaris-server:quarkusAppPartsBuild --rerun \
  -Dquarkus.container-image.build=true

Run the published image:

docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest

The repository also ships docker-compose examples for various configurations. See the Quickstart for details.

Running in Kubernetes

See helm/polaris/README.md.

Configuring Polaris

Servers can be configured in several ways: see the Configuration Guide. Default values live in runtime/defaults/src/main/resources/application.properties.

Building the docs

Docs use Hugo with the Docsy theme. To preview locally:

site/bin/run-hugo-in-docker.sh

See site/README.md for more.

Develocity build scans

Build scans for apache/polaris branch and tag CI runs are published to the ASF Develocity instance at develocity.apache.org when the DEVELOCITY_ACCESS_KEY org secret is available. Pull-request CI publishes to Gradle’s public Develocity instance instead.

Local builds publish a scan only when invoked with --scan, and only after you accept Gradle’s terms of service. Forks and other CI environments can opt in by setting GRADLE_TOS_ACCEPTED=true. Optional overrides: DEVELOCITY_PROJECT_ID, DEVELOCITY_SERVER, and a DEVELOCITY_ACCESS_KEY GitHub secret if you self-host Develocity.

License

Apache Polaris is under the Apache License Version 2.0. See the LICENSE.