This Week in Databend #128
PsiACEJan 15, 2024
Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .
What's New
Stay informed about the latest features of Databend.
Querying Data on HuggingFace File System with Databend
Hugging Face is currently the most popular AI community. Databend now supports direct queries and analysis of massive data and models stored on the Hugging Face file system.
URI format:
hf://{repo_id}/path/to/file
repo_id
fka/awesome-chatgpt-prompts
Supported configurations include:
- : The type of HuggingFace repository, default is
repo_type, available options aredataset,dataset.model - : The revision version of HuggingFace, default is
revision. Can be a branch, tag, or commit in the repository.main - : The API token of HuggingFace.
token
The following example queries
fka/awesome-chatgpt-prompts
If you would like to learn more, please contact the Databend team or refer to the resources listed below:
Code Corner
Discover some fascinating code snippets or projects that showcase our work or learning journey.
Data Type Mappings across Databend, MySQL, and Oracle
This table provides an outline of the mapping of data types between Databend, MySQL, and Oracle.
| Databend | MySQL | Oracle |
|---|---|---|
| TINYINT | TINYINT | NUMBER(3,0) |
| SMALLINT | SMALLINT | NUMBER(5,0) |
| INT | INT | NUMBER(10,0) |
| BIGINT | BIGINT | NUMBER(19,0) |
| FLOAT | FLOAT | FLOAT |
| DOUBLE | DOUBLE | FLOAT(24) |
| DECIMAL | DECIMAL | FLOAT(24) |
| DATE | DATE | DATE |
| TIMESTAMP | TIMESTAMP | NUMBER |
| DATETIME | DATETIME | DATE |
| YEAR | INT | NUMBER |
| VARCHAR | VARCHAR | VARCHAR2 |
| VARCHAR | CHAR | CHAR |
| VARBINARY | VARBINARY | RAW, BLOB |
| VARCHAR | VARCHAR | VARCHAR2 |
| VARCHAR | VARCHAR | RAW, CBLOB |
| VARBINARY | VARBINARY | RAW, BLOB |
| VARCHAR | VARCHAR | RAW, CBLOB |
| VARCHAR | VARCHAR | VARCHAR2 |
| VARCHAR | VARCHAR | VARCHAR2 |
| ARRAY | N/A | N/A |
| BOOLEAN | N/A | N/A |
| TUPLE | N/A | N/A |
| MAP | N/A | N/A |
| JSON, VARIANT | JSON | JSON |
| BITMAP | N/A | N/A |
Highlights
We have also made these improvements to Databend that we hope you will find helpful:
- Added the data type and support for conversion between String and Binary.
Binary - Support for adaptive filter reorder.
- Support for JSON function .
concat - Support for automatic refresh of the table schema.
ReadOnlyAttach - Support for a greedy JOIN order algorithm.
What's Up Next
We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.
Adding Support for Task Advice System Table
Databend plans to introduce the
task_advice
| catalog_name | database_name | table_name | task_type | need_run | task_sql | reason(variant) |
|---|---|---|---|---|---|---|
| default | db | xx | COMPACT | 1 | optimize table xx compact limit 3 | |
| default | db | xx | AGGREGATING_INDEX | 0 | refresh aggregating index xx_agg_idx limit 10 | |
| default | db | yy | ADD_CLUSTER_KEY | 1 | alter table yy cluster by(col1) | |
Issue #14323 | feat: task_advice system table
Please let us know if you're interested in contributing to this feature, or pick up a good first issue at https://link.databend.com/i-m-feeling-lucky to get started.
Changelog
You can check the changelog of Databend Nightly for details about our latest developments.
Full Changelog: https://github.com/datafuselabs/databend/compare/v1.2.286-nightly...v1.2.296-nightly
Subscribe to our newsletter
Stay informed on feature releases, product roadmap, support, and cloud offerings!
