Blog

This Week in Databend #80

PsiACEFeb 8, 2023

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

What's New

Check out what we've done this week to make Databend even better for you.

Features & Improvements ✨

Meta

add databend-meta config
```
grpc_api_advertise_host
```
(#9835)

AST

select from stage with files/pattern (#9877)
parse decimal type (#9894)

Expression

add
```
Decimal128
```
and
```
Decimal256
```
type (#9856)

Functions

support
```
array_indexof
```
(#9840)
support array function
```
array_unique
```
,
```
array_distinct
```
(#9875)
support array aggregate functions (#9903)

Query

add column id in TableSchema; use column id instead of index when read and write data (#9623)
support view in
```
system.columns
```
(#9853)

Storage

```
ParquetTable
```
support topk optimization (#9824)

Sqllogictest

leverage sqllogictest to benchmark tpch (#9887)

Code Refactoring 🎉

Meta

remove obsolete meta service api
```
read_msg()
```
and
```
write_msg()
```
(#9891)

simplify

UserAPI

and

RoleAPI

by introducing a method

update_xx_with(id, f: FnOnce)

(#9921)

Cluster

split exchange source to reader and deserializer (#9805)
split and eliminate the status for exchange transform and sink (#9910)

Functions

rename some array functions add
```
array_
```
prefix (#9886)

Query

```
TableArgs
```
preserve info of positioned and named args (#9917)

Storage

```
ParquetTable
```
list file in
```
read_partition
```
(#9871)

Build/Testing/CI Infra Changes 🔌

support for running benchmark on PRs (#9788)

Bug Fixes 🔧

Functions

fix nullable and or domain cal (#9928)

Planner

fix slow planner when ndv error backtrace (#9876)
fix order by contains aggregation function (#9879)
prevent panic when delete with subquery (#9902)

Query

fix insert default value datatype (#9816)

What's On In Databend

Stay connected with the latest news about Databend.

Why You Should Try Sccache

Sccache is a ccache-like project started by the Mozilla team, supporting C/CPP, Rust and other languages, and storing caches locally or in a cloud storage backend. The community first added native support for the GitHub Action Cache Service to Sccache in version 0.3.3, then improved the functionality in v0.4.0-pre.6 so that the production CI can now use it.

Now, opendal, open-sourced by Datafuse Labs, acts as a storage access layer for sccache to interface with various storage services (s3/gcs/azlob etc.).

Learn More

Blog | Why You Should Try Sccache

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Try using

build-info

To get information about git commits, build options and credits, we now use

vergen

and

cargo-license

build-info can collect build-information of your Rust crate. It might be possible to use it to refactor the relevant logic in

common-building

pub struct BuildInfo {
    pub timestamp: DateTime<Utc>,
    pub profile: String,
    pub optimization_level: OptimizationLevel,
    pub crate_info: CrateInfo,
    pub compiler: CompilerInfo,
    pub version_control: Option<VersionControl>,
}

Issue 9874: Refactor: Try using build-info

Please let us know if you're interested in contributing to this issue, or pick up a good first issue at https://link.databend.com/i-m-feeling-lucky to get started.