> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hydra.so/llms.txt
> Use this file to discover all available pages before exploring further.
# Changelog
* Unlock realtime analytics on existing Postgres deployments
* Unparalleled infrastructure control
* Best price:performance of any analytics database on AWS, GCP, and Azure.
* Automatically managed Hydra versions, build & run dependencies, operating system, and platform architecture using our postgres extension manager, pgxman.
* 1 minute to setup!
**Step 1: Install the Postgres extension manager "pgxman"**\
Curl: `curl -sfL https://install.pgx.sh | sh -`\
Homebrew: `brew install pgxman/tap/pgxman`
**Step 2: Install the Hydra Package**\
`pgxman install hydra_pg_duckdb`
**Step 3: Add configuration for Hydra to your Postgres config file at**: `/etc/postgresql/{version}/main/postgresql.conf`
Add Setting:\
`shared_preload_libraries = 'pg_duckdb'`\
`duckdb.hydra_token = 'fetch token at http://start.hydra.so/get-started' `
Fetch an access token for free from [the URL](https://platform.hydra.so/get-started) above and paste it in.
**Step 4**: Restart Postgres and Create extension\
`sudo service postgresql restart`\
\
`create extension pg_duckdb;`
🎉 Installation Complete! Next, check out the [Quick Start](https://docs.hydra.so/intro/quickstart) guide to start using Serverless Analytics on Postgres.
Overall: [tinyurl.com/benchHydra](http://tinyurl.com/benchHydra)
Hydra transforms Postgres into one of the fastest analytics databases on earth.
| Database Name | Relative time (lower is better) |
| --------------------- | ------------------------------- |
| **Hydra** | **1.51** |
| ClickHouse (c6a.4XL) | 1.93 |
| Snowflake (128x4XL) | 2.71 |
| Redshift (serverless) | 2.80 |
| TimescaleDB | 16.76 |
| AlloyDB | 34.52 |
| AWS Aurora | 277.78 |
| PostgreSQL (indexes) | 35.20 |
| Postgres (no indexes) | 2283 |
Why use the Hydra Package for Postgres? It's free, local, and takes \~10 seconds to setup. Create an analytics table in Postgres: perfect for events, time series, logs, traces.
**python package (**[**pypi**](https://pypi.org/project/hydra-cli/0.0.1/)**)**
Installing hydra-cli enables serverless analytics, Hydra Deep Storage, and more.
By separating compute from storage, Hydra enables bottomless storage for Postgres. Fully integrated columnstore with autogrow, advanced compression, automatic caching, and multi-node reads.
**Features**
* **Storage Auto Grow -** Never worry about running out of bottomless analytics storage.
* **10X Data Compression** - Data stored in analytics tables benefit from efficient data compression of 5-15X and is ideal for large data volumes. For example: 150GB becomes 15GB with a 10X compression.
* **Automatic Caching** - Managed within Hydra to enable sub-second analytics, predictably.
* **Multi-node Reads** - Connect n# of Postgres databases with Hydra to Deep Storage. Ideal for organizations that read events, clicks, traces, time series data from a global source of truth.
By separating compute from storage, Hydra enables compute-isolated analytics. It is designed for low latency applications built on time series and event data.
**Features**
* **Parallel, vectorized execution (**[**benchmark**](https://benchmark.clickhouse.com/#eyJzeXN0ZW0iOnsiQWxsb3lEQiI6ZmFsc2UsIkFsbG95REIgKHR1bmVkKSI6ZmFsc2UsIkF0aGVuYSAocGFydGl0aW9uZWQpIjpmYWxzZSwiQXRoZW5hIChzaW5nbGUpIjpmYWxzZSwiQXVyb3JhIGZvciBNeVNRTCI6ZmFsc2UsIkF1cm9yYSBmb3IgUG9zdGdyZVNRTCI6ZmFsc2UsIkJ5Q29uaXR5IjpmYWxzZSwiQnl0ZUhvdXNlIjpmYWxzZSwiY2hEQiAoRGF0YUZyYW1lKSI6ZmFsc2UsImNoREIgKFBhcnF1ZXQsIHBhcnRpdGlvbmVkKSI6ZmFsc2UsImNoREIiOmZhbHNlLCJDaXR1cyI6ZmFsc2UsIkNsaWNrSG91c2UgQ2xvdWQgKGF3cykiOmZhbHNlLCJDbGlja0hvdXNlIENsb3VkIChhenVyZSkiOmZhbHNlLCJDbGlja0hvdXNlIENsb3VkIChnY3ApIjpmYWxzZSwiQ2xpY2tIb3VzZSAoZGF0YSBsYWtlLCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJDbGlja0hvdXNlIChkYXRhIGxha2UsIHNpbmdsZSkiOmZhbHNlLCJDbGlja0hvdXNlIChQYXJxdWV0LCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJDbGlja0hvdXNlIChQYXJxdWV0LCBzaW5nbGUpIjpmYWxzZSwiQ2xpY2tIb3VzZSAod2ViKSI6ZmFsc2UsIkNsaWNrSG91c2UiOnRydWUsIkNsaWNrSG91c2UgKHR1bmVkKSI6ZmFsc2UsIkNsaWNrSG91c2UgKHR1bmVkLCBtZW1vcnkpIjpmYWxzZSwiQ2xvdWRiZXJyeSI6ZmFsc2UsIkNyYXRlREIiOmZhbHNlLCJDcnVuY2h5IEJyaWRnZSBmb3IgQW5hbHl0aWNzIChQYXJxdWV0KSI6ZmFsc2UsIkRhdGFiZW5kIjpmYWxzZSwiRGF0YUZ1c2lvbiAoUGFycXVldCwgcGFydGl0aW9uZWQpIjpmYWxzZSwiRGF0YUZ1c2lvbiAoUGFycXVldCwgc2luZ2xlKSI6ZmFsc2UsIkFwYWNoZSBEb3JpcyI6ZmFsc2UsIkRyaWxsIjpmYWxzZSwiRHJ1aWQiOmZhbHNlLCJEdWNrREIgKERhdGFGcmFtZSkiOmZhbHNlLCJEdWNrREIgKG1lbW9yeSkiOmZhbHNlLCJEdWNrREIgKFBhcnF1ZXQsIHBhcnRpdGlvbmVkKSI6ZmFsc2UsIkR1Y2tEQiI6ZmFsc2UsIkVsYXN0aWNzZWFyY2giOmZhbHNlLCJFbGFzdGljc2VhcmNoICh0dW5lZCkiOmZhbHNlLCJHbGFyZURCIjpmYWxzZSwiR3JlZW5wbHVtIjp0cnVlLCJIZWF2eUFJIjpmYWxzZSwiSHlkcmEiOnRydWUsIlNhbGVzZm9yY2UgSHlwZXIgKFBhcnF1ZXQpIjpmYWxzZSwiU2FsZXNmb3JjZSBIeXBlciI6ZmFsc2UsIkluZm9icmlnaHQiOmZhbHNlLCJLaW5ldGljYSI6ZmFsc2UsIk1hcmlhREIgQ29sdW1uU3RvcmUiOmZhbHNlLCJNYXJpYURCIjpmYWxzZSwiTW9uZXREQiI6ZmFsc2UsIk1vbmdvREIiOmZhbHNlLCJNb3RoZXJEdWNrIjpmYWxzZSwiTXlTUUwgKE15SVNBTSkiOmZhbHNlLCJNeVNRTCI6ZmFsc2UsIk9jdG9TUUwiOmZhbHNlLCJPcHRlcnl4IjpmYWxzZSwiT3hsYSI6ZmFsc2UsIlBhbmRhcyAoRGF0YUZyYW1lKSI6ZmFsc2UsIlBhcmFkZURCIChQYXJxdWV0LCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJQYXJhZGVEQiAoUGFycXVldCwgc2luZ2xlKSI6ZmFsc2UsInBnX2R1Y2tkYiAod2l0aCBpbmRleGVzKSI6ZmFsc2UsInBnX2R1Y2tkYiAoTW90aGVyRHVjayBlbmFibGVkKSI6ZmFsc2UsInBnX2R1Y2tkYiI6ZmFsc2UsInBnX2R1Y2tkYiAoUGFycXVldCkiOmZhbHNlLCJQb3N0Z3JlU1FMIHdpdGggcGdfbW9vbmNha2UiOmZhbHNlLCJQaW5vdCI6ZmFsc2UsIlBvbGFycyAoRGF0YUZyYW1lKSI6ZmFsc2UsIlBvbGFycyAoUGFycXVldCkiOmZhbHNlLCJQb3N0Z3JlU1FMICh3aXRoIGluZGV4ZXMpIjpmYWxzZSwiUG9zdGdyZVNRTCI6ZmFsc2UsIlF1ZXN0REIiOmZhbHNlLCJSZWRzaGlmdCI6dHJ1ZSwiU2VsZWN0REIiOmZhbHNlLCJTaW5nbGVTdG9yZSI6ZmFsc2UsIlNub3dmbGFrZSI6dHJ1ZSwiU3BhcmsiOmZhbHNlLCJTUUxpdGUiOmZhbHNlLCJTdGFyUm9ja3MiOmZhbHNlLCJUYWJsZXNwYWNlIjpmYWxzZSwiVGVtYm8gT0xBUCAoY29sdW1uYXIpIjpmYWxzZSwiVGltZXNjYWxlIENsb3VkIjp0cnVlLCJUaW1lc2NhbGVEQiAobm8gY29sdW1uc3RvcmUpIjpmYWxzZSwiVGltZXNjYWxlREIiOmZhbHNlLCJUaW55YmlyZCAoRnJlZSBUcmlhbCkiOnRydWUsIlVtYnJhIjpmYWxzZSwiVXJzYSI6ZmFsc2UsIlZpY3RvcmlhTG9ncyI6ZmFsc2V9LCJ0eXBlIjp7IkMiOnRydWUsImNvbHVtbi1vcmllbnRlZCI6dHJ1ZSwiUG9zdGdyZVNRTCBjb21wYXRpYmxlIjp0cnVlLCJtYW5hZ2VkIjp0cnVlLCJnY3AiOnRydWUsInN0YXRlbGVzcyI6dHJ1ZSwiSmF2YSI6dHJ1ZSwiQysrIjp0cnVlLCJNeVNRTCBjb21wYXRpYmxlIjp0cnVlLCJyb3ctb3JpZW50ZWQiOnRydWUsIkNsaWNrSG91c2UgZGVyaXZhdGl2ZSI6dHJ1ZSwiZW1iZWRkZWQiOnRydWUsInNlcnZlcmxlc3MiOnRydWUsImRhdGFmcmFtZSI6dHJ1ZSwiYXdzIjp0cnVlLCJhenVyZSI6dHJ1ZSwiYW5hbHl0aWNhbCI6dHJ1ZSwiUnVzdCI6dHJ1ZSwic2VhcmNoIjp0cnVlLCJkb2N1bWVudCI6dHJ1ZSwiR28iOnRydWUsInNvbWV3aGF0IFBvc3RncmVTUUwgY29tcGF0aWJsZSI6dHJ1ZSwiRGF0YUZyYW1lIjp0cnVlLCJwYXJxdWV0Ijp0cnVlLCJ0aW1lLXNlcmllcyI6dHJ1ZX0sIm1hY2hpbmUiOnsiMTYgdkNQVSAxMjhHQiI6dHJ1ZSwiOCB2Q1BVIDY0R0IiOmZhbHNlLCJzZXJ2ZXJsZXNzIjp0cnVlLCIxNmFjdSI6dHJ1ZSwiYzZhLjR4bGFyZ2UsIDUwMGdiIGdwMiI6dHJ1ZSwiTCI6dHJ1ZSwiTSI6dHJ1ZSwiUyI6dHJ1ZSwiWFMiOmZhbHNlLCJjNmEubWV0YWwsIDUwMGdiIGdwMiI6ZmFsc2UsIjEyR2lCLCAxIHJlcGxpY2EocykiOnRydWUsIjhHaUIsIDEgcmVwbGljYShzKSI6dHJ1ZSwiMTJHaUIsIDIgcmVwbGljYShzKSI6dHJ1ZSwiMTIwR2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjE2R2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjIzNkdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCIzMkdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCI2NEdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCI4R2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjEyR2lCLCAzIHJlcGxpY2EocykiOnRydWUsIjEyMEdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCIxNkdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCIyMzZHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiMzJHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiNjRHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiOEdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCJjNW4uNHhsYXJnZSwgNTAwZ2IgZ3AyIjp0cnVlLCJBbmFseXRpY3MtMjU2R0IgKDY0IHZDb3JlcywgMjU2IEdCKSI6dHJ1ZSwiYzUuNHhsYXJnZSwgNTAwZ2IgZ3AyIjp0cnVlLCJjNmEuNHhsYXJnZSwgMTUwMGdiIGdwMiI6dHJ1ZSwiWEwiOnRydWUsIkp1bWJvIjp0cnVlLCJQdWxzZSI6dHJ1ZSwiU3RhbmRhcmQiOnRydWUsImRjMi44eGxhcmdlIjpmYWxzZSwicmEzLjE2eGxhcmdlIjpmYWxzZSwicmEzLjR4bGFyZ2UiOmZhbHNlLCJyYTMueGxwbHVzIjpmYWxzZSwiUzIiOnRydWUsIlMyNCI6dHJ1ZSwiMlhMIjp0cnVlLCIzWEwiOnRydWUsIjRYTCI6dHJ1ZSwiTDEgLSAxNkNQVSAzMkdCIjp0cnVlLCJjNmEuNHhsYXJnZSwgNTAwZ2IgZ3AzIjp0cnVlLCIxNiB2Q1BVIDY0R0IiOnRydWUsIjQgdkNQVSAxNkdCIjpmYWxzZSwiOCB2Q1BVIDMyR0IiOmZhbHNlfSwiY2x1c3Rlcl9zaXplIjp7IjEiOnRydWUsIjIiOmZhbHNlLCIzIjp0cnVlLCI0IjpmYWxzZSwiOCI6ZmFsc2UsIjE2IjpmYWxzZSwiMzIiOmZhbHNlLCI2NCI6ZmFsc2UsIjEyOCI6dHJ1ZSwic2VydmVybGVzcyI6dHJ1ZSwidW5kZWZpbmVkIjp0cnVlfSwibWV0cmljIjoiaG90IiwicXVlcmllcyI6W3RydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWVdfQ==)**)**
Example Performance Results ( Hydra vs. PostgreSQL)
* **Compute Autoscale** - Hydra scales compute automatically per query to ensure preditably quick efficient processing of analytical queries.
* **Isolated Compute Tenancy** - Processes run on dedicated compute resources, improving performance by preventing resource contention. This update enhances performance reliability and preditability through scale.
**Added**
* Support using Postgres indexes and reading from partitioned tables.
* The `AS (id bigint, name text)` syntax is no longer supported when using `read_parquet`, `iceberg_scan`, etc. The new syntax is as follows:
```sql theme={null}
SELECT * FROM read_parquet('file.parquet');
SELECT r['id'], r['name'] FROM read_parquet('file.parquet') r WHERE r['age'] > 21;
```
* Add a `duckdb.query` function which allows using DuckDB query syntax in Postgres.
* Support the `approx_count_distinct` DuckDB aggregate.
* Support the `bytea` (aka blob), `uhugeint`,`jsonb`, `timestamp_ns`, `timestamp_ms`, `timestamp_s` & `interval` types.
* Support DuckDB [json functions and aggregates](https://duckdb.org/docs/data/json/json_functions.html).
* Add support for the `duckdb.allow_community_extensions` setting.
* We have an official logo\\\\\\! 🎉
**Changed**
* Allow executing `duckdb.raw_query`, `duckdb.cache_info`, `duckdb.cache_delete` and `duckdb.recycle_db` as non-superusers.
**Fixed**
* Correctly parse parameter lists in `COPY` commands. This allows using `PARTITION_BY` as one of the `COPY` options.
* Correctly read cache metadata for files larger than 4GB.
* Fix bug in parameter handling for prepared statements and PL/pgSQL functions.
* Fix comparisons and operators on the `timestamp with timezone` field by enabling DuckDB its `icu` extension by default.
* Allow using `read_parquet` functions when not using superuser privileges.
* Fix some case insensitivity issues when reading from Postgres tables.
**Added**
* Support for reading Delta Lake storage using the `duckdb.delta_scan(...)` function.
* Support for reading JSON using the `duckdb.read_json(...)` function.
* Support for multi-statement transactions.
* Support reading from Azure Blob storage.
* Support many more array types, such as `float` , `numeric` and `uuid` arrays.
* Support for PostgreSQL 14.
* Manage cached files using the `duckdb.cache_info()` and `duckdb.cache_delete()` functions.
* Add `scope` column to `duckdb.secrets` table.
* Automatically install and load known DuckDB extensions when queries use them. So, `duckdb.install_extension()` is usually not necessary anymore.
**Changed**
* Improve performance of heap reading.
* Bump DuckDB version to 1.1.3.
**Fixed**
* Throw a clear error when reading partitioned tables (reading from partitioned tables is not supported yet).
* Fixed crash when using `CREATE SCHEMA AUTHORIZATION`.
* Fix queries inserting into DuckDB tables with `DEFAULT` values.
* Fixed assertion failure involving recursive CTEs.
* Much better separation between C and C++ code, to avoid memory leaks and crashes (many PRs).
* `SELECT` queries executed by the DuckDB engine can directly read Postgres tables. (If you only query Postgres tables you need to run `SET duckdb.force_execution TO true`, see the **IMPORTANT** section above for details)
* Able to read [data types](https://www.postgresql.org/docs/current/datatype.html) that exist in both Postgres and DuckDB.
* If DuckDB cannot support the query for any reason, execution falls back to Postgres.
* Read and Write support for object storage (AWS S3, Azure, Cloudflare R2, or Google GCS):
* Read parquet, CSV and JSON files:
* `SELECT n FROM read_parquet('s3://bucket/file.parquet') AS (n int)`
* `SELECT n FROM read_csv('s3://bucket/file.csv') AS (n int)`
* You can pass globs and arrays to these functions, just like in DuckDB
* Enable the DuckDB Iceberg extension using `SELECT duckdb.install_extension('iceberg')` and read Iceberg files with `iceberg_scan`.
* Write a query — or an entire table — to parquet in object storage.
* `COPY (SELECT foo, bar FROM baz) TO 's3://...'`
* `COPY table TO 's3://...'`
* Read and write to Parquet format in a single query COPY ( SELECT count(\*), name FROM read\_parquet('s3://bucket/file.parquet') AS (name text) GROUP BY name ORDER BY count DESC ) TO 's3://bucket/results.parquet';
* Query and `JOIN` data in object storage/MotherDuck with Postgres tables, views, and materialized views.
* Create temporary tables in DuckDB its columnar storage format using `CREATE TEMP TABLE ... USING duckdb`.
* Install DuckDB extensions using `SELECT duckdb.install_extension('extension_name');`
* Toggle DuckDB execution on/off with a setting:
* `SET duckdb.force_execution = true|false`
* Cache remote object locally for faster execution using `SELECT duckdb.cache('path', 'type');` where
* 'path' is HTTPFS/S3/GCS/R2 remote object
* 'type' specify remote object type: 'parquet' or 'csv'
Overall: [tinyurl.com/benchHydra](http://tinyurl.com/benchHydra)
Hydra transforms Postgres into one of the fastest analytics databases on earth.
| Database Name | Relative time (lower is better) |
| --------------------- | ------------------------------- |
| **Hydra** | **1.51** |
| ClickHouse (c6a.4XL) | 1.93 |
| Snowflake (128x4XL) | 2.71 |
| Redshift (serverless) | 2.80 |
| TimescaleDB | 16.76 |
| AlloyDB | 34.52 |
| AWS Aurora | 277.78 |
| PostgreSQL (indexes) | 35.20 |
| Postgres (no indexes) | 2283 |
Why use the Hydra Package for Postgres? It's free, local, and takes \~10 seconds to setup. Create an analytics table in Postgres: perfect for events, time series, logs, traces.
**python package (**[**pypi**](https://pypi.org/project/hydra-cli/0.0.1/)**)**
Installing hydra-cli enables serverless analytics, Hydra Deep Storage, and more.
By separating compute from storage, Hydra enables bottomless storage for Postgres. Fully integrated columnstore with autogrow, advanced compression, automatic caching, and multi-node reads.
**Features**
* **Storage Auto Grow -** Never worry about running out of bottomless analytics storage.
* **10X Data Compression** - Data stored in analytics tables benefit from efficient data compression of 5-15X and is ideal for large data volumes. For example: 150GB becomes 15GB with a 10X compression.
* **Automatic Caching** - Managed within Hydra to enable sub-second analytics, predictably.
* **Multi-node Reads** - Connect n# of Postgres databases with Hydra to Deep Storage. Ideal for organizations that read events, clicks, traces, time series data from a global source of truth.
By separating compute from storage, Hydra enables compute-isolated analytics. It is designed for low latency applications built on time series and event data.
**Features**
* **Parallel, vectorized execution (**[**benchmark**](https://benchmark.clickhouse.com/#eyJzeXN0ZW0iOnsiQWxsb3lEQiI6ZmFsc2UsIkFsbG95REIgKHR1bmVkKSI6ZmFsc2UsIkF0aGVuYSAocGFydGl0aW9uZWQpIjpmYWxzZSwiQXRoZW5hIChzaW5nbGUpIjpmYWxzZSwiQXVyb3JhIGZvciBNeVNRTCI6ZmFsc2UsIkF1cm9yYSBmb3IgUG9zdGdyZVNRTCI6ZmFsc2UsIkJ5Q29uaXR5IjpmYWxzZSwiQnl0ZUhvdXNlIjpmYWxzZSwiY2hEQiAoRGF0YUZyYW1lKSI6ZmFsc2UsImNoREIgKFBhcnF1ZXQsIHBhcnRpdGlvbmVkKSI6ZmFsc2UsImNoREIiOmZhbHNlLCJDaXR1cyI6ZmFsc2UsIkNsaWNrSG91c2UgQ2xvdWQgKGF3cykiOmZhbHNlLCJDbGlja0hvdXNlIENsb3VkIChhenVyZSkiOmZhbHNlLCJDbGlja0hvdXNlIENsb3VkIChnY3ApIjpmYWxzZSwiQ2xpY2tIb3VzZSAoZGF0YSBsYWtlLCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJDbGlja0hvdXNlIChkYXRhIGxha2UsIHNpbmdsZSkiOmZhbHNlLCJDbGlja0hvdXNlIChQYXJxdWV0LCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJDbGlja0hvdXNlIChQYXJxdWV0LCBzaW5nbGUpIjpmYWxzZSwiQ2xpY2tIb3VzZSAod2ViKSI6ZmFsc2UsIkNsaWNrSG91c2UiOnRydWUsIkNsaWNrSG91c2UgKHR1bmVkKSI6ZmFsc2UsIkNsaWNrSG91c2UgKHR1bmVkLCBtZW1vcnkpIjpmYWxzZSwiQ2xvdWRiZXJyeSI6ZmFsc2UsIkNyYXRlREIiOmZhbHNlLCJDcnVuY2h5IEJyaWRnZSBmb3IgQW5hbHl0aWNzIChQYXJxdWV0KSI6ZmFsc2UsIkRhdGFiZW5kIjpmYWxzZSwiRGF0YUZ1c2lvbiAoUGFycXVldCwgcGFydGl0aW9uZWQpIjpmYWxzZSwiRGF0YUZ1c2lvbiAoUGFycXVldCwgc2luZ2xlKSI6ZmFsc2UsIkFwYWNoZSBEb3JpcyI6ZmFsc2UsIkRyaWxsIjpmYWxzZSwiRHJ1aWQiOmZhbHNlLCJEdWNrREIgKERhdGFGcmFtZSkiOmZhbHNlLCJEdWNrREIgKG1lbW9yeSkiOmZhbHNlLCJEdWNrREIgKFBhcnF1ZXQsIHBhcnRpdGlvbmVkKSI6ZmFsc2UsIkR1Y2tEQiI6ZmFsc2UsIkVsYXN0aWNzZWFyY2giOmZhbHNlLCJFbGFzdGljc2VhcmNoICh0dW5lZCkiOmZhbHNlLCJHbGFyZURCIjpmYWxzZSwiR3JlZW5wbHVtIjp0cnVlLCJIZWF2eUFJIjpmYWxzZSwiSHlkcmEiOnRydWUsIlNhbGVzZm9yY2UgSHlwZXIgKFBhcnF1ZXQpIjpmYWxzZSwiU2FsZXNmb3JjZSBIeXBlciI6ZmFsc2UsIkluZm9icmlnaHQiOmZhbHNlLCJLaW5ldGljYSI6ZmFsc2UsIk1hcmlhREIgQ29sdW1uU3RvcmUiOmZhbHNlLCJNYXJpYURCIjpmYWxzZSwiTW9uZXREQiI6ZmFsc2UsIk1vbmdvREIiOmZhbHNlLCJNb3RoZXJEdWNrIjpmYWxzZSwiTXlTUUwgKE15SVNBTSkiOmZhbHNlLCJNeVNRTCI6ZmFsc2UsIk9jdG9TUUwiOmZhbHNlLCJPcHRlcnl4IjpmYWxzZSwiT3hsYSI6ZmFsc2UsIlBhbmRhcyAoRGF0YUZyYW1lKSI6ZmFsc2UsIlBhcmFkZURCIChQYXJxdWV0LCBwYXJ0aXRpb25lZCkiOmZhbHNlLCJQYXJhZGVEQiAoUGFycXVldCwgc2luZ2xlKSI6ZmFsc2UsInBnX2R1Y2tkYiAod2l0aCBpbmRleGVzKSI6ZmFsc2UsInBnX2R1Y2tkYiAoTW90aGVyRHVjayBlbmFibGVkKSI6ZmFsc2UsInBnX2R1Y2tkYiI6ZmFsc2UsInBnX2R1Y2tkYiAoUGFycXVldCkiOmZhbHNlLCJQb3N0Z3JlU1FMIHdpdGggcGdfbW9vbmNha2UiOmZhbHNlLCJQaW5vdCI6ZmFsc2UsIlBvbGFycyAoRGF0YUZyYW1lKSI6ZmFsc2UsIlBvbGFycyAoUGFycXVldCkiOmZhbHNlLCJQb3N0Z3JlU1FMICh3aXRoIGluZGV4ZXMpIjpmYWxzZSwiUG9zdGdyZVNRTCI6ZmFsc2UsIlF1ZXN0REIiOmZhbHNlLCJSZWRzaGlmdCI6dHJ1ZSwiU2VsZWN0REIiOmZhbHNlLCJTaW5nbGVTdG9yZSI6ZmFsc2UsIlNub3dmbGFrZSI6dHJ1ZSwiU3BhcmsiOmZhbHNlLCJTUUxpdGUiOmZhbHNlLCJTdGFyUm9ja3MiOmZhbHNlLCJUYWJsZXNwYWNlIjpmYWxzZSwiVGVtYm8gT0xBUCAoY29sdW1uYXIpIjpmYWxzZSwiVGltZXNjYWxlIENsb3VkIjp0cnVlLCJUaW1lc2NhbGVEQiAobm8gY29sdW1uc3RvcmUpIjpmYWxzZSwiVGltZXNjYWxlREIiOmZhbHNlLCJUaW55YmlyZCAoRnJlZSBUcmlhbCkiOnRydWUsIlVtYnJhIjpmYWxzZSwiVXJzYSI6ZmFsc2UsIlZpY3RvcmlhTG9ncyI6ZmFsc2V9LCJ0eXBlIjp7IkMiOnRydWUsImNvbHVtbi1vcmllbnRlZCI6dHJ1ZSwiUG9zdGdyZVNRTCBjb21wYXRpYmxlIjp0cnVlLCJtYW5hZ2VkIjp0cnVlLCJnY3AiOnRydWUsInN0YXRlbGVzcyI6dHJ1ZSwiSmF2YSI6dHJ1ZSwiQysrIjp0cnVlLCJNeVNRTCBjb21wYXRpYmxlIjp0cnVlLCJyb3ctb3JpZW50ZWQiOnRydWUsIkNsaWNrSG91c2UgZGVyaXZhdGl2ZSI6dHJ1ZSwiZW1iZWRkZWQiOnRydWUsInNlcnZlcmxlc3MiOnRydWUsImRhdGFmcmFtZSI6dHJ1ZSwiYXdzIjp0cnVlLCJhenVyZSI6dHJ1ZSwiYW5hbHl0aWNhbCI6dHJ1ZSwiUnVzdCI6dHJ1ZSwic2VhcmNoIjp0cnVlLCJkb2N1bWVudCI6dHJ1ZSwiR28iOnRydWUsInNvbWV3aGF0IFBvc3RncmVTUUwgY29tcGF0aWJsZSI6dHJ1ZSwiRGF0YUZyYW1lIjp0cnVlLCJwYXJxdWV0Ijp0cnVlLCJ0aW1lLXNlcmllcyI6dHJ1ZX0sIm1hY2hpbmUiOnsiMTYgdkNQVSAxMjhHQiI6dHJ1ZSwiOCB2Q1BVIDY0R0IiOmZhbHNlLCJzZXJ2ZXJsZXNzIjp0cnVlLCIxNmFjdSI6dHJ1ZSwiYzZhLjR4bGFyZ2UsIDUwMGdiIGdwMiI6dHJ1ZSwiTCI6dHJ1ZSwiTSI6dHJ1ZSwiUyI6dHJ1ZSwiWFMiOmZhbHNlLCJjNmEubWV0YWwsIDUwMGdiIGdwMiI6ZmFsc2UsIjEyR2lCLCAxIHJlcGxpY2EocykiOnRydWUsIjhHaUIsIDEgcmVwbGljYShzKSI6dHJ1ZSwiMTJHaUIsIDIgcmVwbGljYShzKSI6dHJ1ZSwiMTIwR2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjE2R2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjIzNkdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCIzMkdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCI2NEdpQiwgMiByZXBsaWNhKHMpIjp0cnVlLCI4R2lCLCAyIHJlcGxpY2EocykiOnRydWUsIjEyR2lCLCAzIHJlcGxpY2EocykiOnRydWUsIjEyMEdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCIxNkdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCIyMzZHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiMzJHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiNjRHaUIsIDMgcmVwbGljYShzKSI6dHJ1ZSwiOEdpQiwgMyByZXBsaWNhKHMpIjp0cnVlLCJjNW4uNHhsYXJnZSwgNTAwZ2IgZ3AyIjp0cnVlLCJBbmFseXRpY3MtMjU2R0IgKDY0IHZDb3JlcywgMjU2IEdCKSI6dHJ1ZSwiYzUuNHhsYXJnZSwgNTAwZ2IgZ3AyIjp0cnVlLCJjNmEuNHhsYXJnZSwgMTUwMGdiIGdwMiI6dHJ1ZSwiWEwiOnRydWUsIkp1bWJvIjp0cnVlLCJQdWxzZSI6dHJ1ZSwiU3RhbmRhcmQiOnRydWUsImRjMi44eGxhcmdlIjpmYWxzZSwicmEzLjE2eGxhcmdlIjpmYWxzZSwicmEzLjR4bGFyZ2UiOmZhbHNlLCJyYTMueGxwbHVzIjpmYWxzZSwiUzIiOnRydWUsIlMyNCI6dHJ1ZSwiMlhMIjp0cnVlLCIzWEwiOnRydWUsIjRYTCI6dHJ1ZSwiTDEgLSAxNkNQVSAzMkdCIjp0cnVlLCJjNmEuNHhsYXJnZSwgNTAwZ2IgZ3AzIjp0cnVlLCIxNiB2Q1BVIDY0R0IiOnRydWUsIjQgdkNQVSAxNkdCIjpmYWxzZSwiOCB2Q1BVIDMyR0IiOmZhbHNlfSwiY2x1c3Rlcl9zaXplIjp7IjEiOnRydWUsIjIiOmZhbHNlLCIzIjp0cnVlLCI0IjpmYWxzZSwiOCI6ZmFsc2UsIjE2IjpmYWxzZSwiMzIiOmZhbHNlLCI2NCI6ZmFsc2UsIjEyOCI6dHJ1ZSwic2VydmVybGVzcyI6dHJ1ZSwidW5kZWZpbmVkIjp0cnVlfSwibWV0cmljIjoiaG90IiwicXVlcmllcyI6W3RydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWUsdHJ1ZSx0cnVlLHRydWVdfQ==)**)**
.png?fit=max&auto=format&n=qsYyo-rksckbkjwh&q=85&s=766329ffb359ba92db9b1916c124f600)
Example Performance Results ( Hydra vs. PostgreSQL)
* **Compute Autoscale** - Hydra scales compute automatically per query to ensure preditably quick efficient processing of analytical queries.
* **Isolated Compute Tenancy** - Processes run on dedicated compute resources, improving performance by preventing resource contention. This update enhances performance reliability and preditability through scale.
**Added**
* Support using Postgres indexes and reading from partitioned tables.
* The `AS (id bigint, name text)` syntax is no longer supported when using `read_parquet`, `iceberg_scan`, etc. The new syntax is as follows:
```sql theme={null}
SELECT * FROM read_parquet('file.parquet');
SELECT r['id'], r['name'] FROM read_parquet('file.parquet') r WHERE r['age'] > 21;
```
* Add a `duckdb.query` function which allows using DuckDB query syntax in Postgres.
* Support the `approx_count_distinct` DuckDB aggregate.
* Support the `bytea` (aka blob), `uhugeint`,`jsonb`, `timestamp_ns`, `timestamp_ms`, `timestamp_s` & `interval` types.
* Support DuckDB [json functions and aggregates](https://duckdb.org/docs/data/json/json_functions.html).
* Add support for the `duckdb.allow_community_extensions` setting.
* We have an official logo\\\\\\! 🎉
**Changed**
* Allow executing `duckdb.raw_query`, `duckdb.cache_info`, `duckdb.cache_delete` and `duckdb.recycle_db` as non-superusers.
**Fixed**
* Correctly parse parameter lists in `COPY` commands. This allows using `PARTITION_BY` as one of the `COPY` options.
* Correctly read cache metadata for files larger than 4GB.
* Fix bug in parameter handling for prepared statements and PL/pgSQL functions.
* Fix comparisons and operators on the `timestamp with timezone` field by enabling DuckDB its `icu` extension by default.
* Allow using `read_parquet` functions when not using superuser privileges.
* Fix some case insensitivity issues when reading from Postgres tables.
**Added**
* Support for reading Delta Lake storage using the `duckdb.delta_scan(...)` function.
* Support for reading JSON using the `duckdb.read_json(...)` function.
* Support for multi-statement transactions.
* Support reading from Azure Blob storage.
* Support many more array types, such as `float` , `numeric` and `uuid` arrays.
* Support for PostgreSQL 14.
* Manage cached files using the `duckdb.cache_info()` and `duckdb.cache_delete()` functions.
* Add `scope` column to `duckdb.secrets` table.
* Automatically install and load known DuckDB extensions when queries use them. So, `duckdb.install_extension()` is usually not necessary anymore.
**Changed**
* Improve performance of heap reading.
* Bump DuckDB version to 1.1.3.
**Fixed**
* Throw a clear error when reading partitioned tables (reading from partitioned tables is not supported yet).
* Fixed crash when using `CREATE SCHEMA AUTHORIZATION`.
* Fix queries inserting into DuckDB tables with `DEFAULT` values.
* Fixed assertion failure involving recursive CTEs.
* Much better separation between C and C++ code, to avoid memory leaks and crashes (many PRs).
* `SELECT` queries executed by the DuckDB engine can directly read Postgres tables. (If you only query Postgres tables you need to run `SET duckdb.force_execution TO true`, see the **IMPORTANT** section above for details)
* Able to read [data types](https://www.postgresql.org/docs/current/datatype.html) that exist in both Postgres and DuckDB.
* If DuckDB cannot support the query for any reason, execution falls back to Postgres.
* Read and Write support for object storage (AWS S3, Azure, Cloudflare R2, or Google GCS):
* Read parquet, CSV and JSON files:
* `SELECT n FROM read_parquet('s3://bucket/file.parquet') AS (n int)`
* `SELECT n FROM read_csv('s3://bucket/file.csv') AS (n int)`
* You can pass globs and arrays to these functions, just like in DuckDB
* Enable the DuckDB Iceberg extension using `SELECT duckdb.install_extension('iceberg')` and read Iceberg files with `iceberg_scan`.
* Write a query — or an entire table — to parquet in object storage.
* `COPY (SELECT foo, bar FROM baz) TO 's3://...'`
* `COPY table TO 's3://...'`
* Read and write to Parquet format in a single query COPY ( SELECT count(\*), name FROM read\_parquet('s3://bucket/file.parquet') AS (name text) GROUP BY name ORDER BY count DESC ) TO 's3://bucket/results.parquet';
* Query and `JOIN` data in object storage/MotherDuck with Postgres tables, views, and materialized views.
* Create temporary tables in DuckDB its columnar storage format using `CREATE TEMP TABLE ... USING duckdb`.
* Install DuckDB extensions using `SELECT duckdb.install_extension('extension_name');`
* Toggle DuckDB execution on/off with a setting:
* `SET duckdb.force_execution = true|false`
* Cache remote object locally for faster execution using `SELECT duckdb.cache('path', 'type');` where
* 'path' is HTTPFS/S3/GCS/R2 remote object
* 'type' specify remote object type: 'parquet' or 'csv'