This Week in Databend #84

Databend is a modern cloud data warehouse, serving your massive-scale analytics needs at low cost and complexity. Open source alternative to Snowflake. Also available in the cloud: https://app.databend.com .

📢 We announced the milestone release of Databend 1.0 this week. For what's new in the release, please read this post: https://www.databend.com/blog/databend-release-v1.0 .

What's On In Databend

Stay connected with the latest news about Databend.

SQL:

REPLACE INTO

The

REPLACE INTO

If a row with the same conflict key already exists in the table, Databend will update the row with the new data. Otherwise, the new data will be added to the table as a new row. You can use this statement to easily sync data from different sources or handle duplicate records.

#> CREATE TABLE employees(id INT, name VARCHAR, salary INT);
#> REPLACE INTO employees (id, name, salary) ON (id) VALUES (1, 'John Doe', 50000);
#> SELECT  * FROM Employees;
+------+----------+--------+
| id   | name     | salary |
+------+----------+--------+
|    1 | John Doe |  50000 |
+------+----------+--------+

If you want to learn more details about

REPLACE INTO

• Docs | DML Commands - REPLACE
• PR | feat: replace into statement

RFC: Add Incremental Update for Copy Into

Databend is currently capable of transforming and inserting data from a stage into a table. For example, you can run a SQL statement like this:

insert into table1 from (select c1， c2 from @stage1/path/to/dir);

The COPY INTO command needs a similar feature as well to support incremental data loading from a stage.

If you're interested, check the following RFCs:

• Docs | RFCs - Transform During Load
• Docs | RFCs - Stage With Schema

Code Corner

Discover some fascinating code snippets or projects that showcase our work or learning journey.

Rust Toolchain: 🦀 v1.70.0-nightly (03-10)

Databend has upgraded its Rust toolchain to

nightly-2023-03-10

cargo clean

We also fixed some clippy warnings. We now use

#[default]

enum

#[derive(Debug, Default)]
pub enum RecordDelimiter {
    #[default]
    Crlf,
    Any(u8),
}

If you are interested in this Rust trick, you can read this RFC:

derive_default_enum

Announcing OpenDAL v0.30

OpenDAL brings some important changes in its newly released 0.30 version, including removing

Object

Removing

Object

The term

Object

opendal::Object

Object

Operator

# get metadata of a path
- op.object("path").stat().await;
+ op.stat("path").await;

Bindings for JavaScript and Python

We have released OpenDAL's JavaScript and Python bindings, thanks to @suyanhanx, @messense, @Xuanwo, and others who were involved, Now, users of these languages can access data from various services using OpenDAL. This is especially useful for data scientists and data analysts who only require OpenDAL and do not need any other SDKs.

Here is a Python example:

>>> import opendal
>>> op = opendal.Operator('memory')
>>> op.write("test", b"Hello, World!")
>>> op.read("test")
b'Hello, World!'

What's Up Next

We're always open to cutting-edge technologies and innovative ideas. You're more than welcome to join the community and bring them to Databend.

Support More Mathematical Operators like PostgreSQL

More mathematical operators in SQL expressions are required to perform complex calculations instead of writing them in another language or using external tools.

@jun0315 is working on it and has made some great progress. If you are interested in this feature, feel free to drop in and contribute.

Issue 10233: Feature: support more pg functions

Please let us know if you're interested in contributing to this issue, or pick up a good first issue at https://link.databend.com/i-m-feeling-lucky to get started.

New Contributors

We always open arms to everyone and can't wait to see how you'll help our community grow and thrive.

• @jun0315 made their first contribution in #10347, adding the

  caret
  
  function that performs exponentiation. For example, 2^3 means 2 raised to the power of 3, which equals 8.
• @quaxquax made their first contribution in #10465, fixing a typo, thanks!

Changelog

You can check the changelog of Databend Nightly for details about our latest developments.

Full Changelog: https://github.com/databendlabs/databend/compare/v1.0.1-nightly...v1.0.10-nightly