Skip to content

[Bug] sequence列删除成功之后,无法新增一模一样的数据 #17739

Closed

Description

@dearMOMO

Search before asking

  • I had searched in the issues and found no similar issues.

Version

1.2.2

What's Wrong?

你好
这边发现一个问题(不确定是不是BUG)。使用的是主键模型【当存在sequence列时,将与导入数据key 相同的数据全部删除】;删除数据时按照sequence列进行删除。删除成功之后,我想恢复这条数据,于是我又重新同步了一条相同的一份数据,这条数据无法新增成功,streamload返回成功;可是当我把sequence列字段的值设置的比原先大的时候,又可以新增数据;

这个是分两次执行的事务的, 删除了一条数据,然后又新增了一条一模一样的数据,结果看不到

What You Expected?

被删除的数据,我新增相同的数据,依然可以查询出来

How to Reproduce?

1.项目运行地址

https://gitee.com/momoriven/ApacheDoris.git

2.创建数据库
CREATE DATABASE example_db;

3.执行建表语句

CREATE TABLE IF NOT EXISTS example_db.unique_detail (
user_id VARCHAR(30) NOT NULL COMMENT "用户id",
register_time DATE COMMENT "用户注册时间",
username VARCHAR(50) COMMENT "用户昵称",
city VARCHAR(20) COMMENT "用户所在城市",
age SMALLINT COMMENT "用户年龄",
sex TINYINT COMMENT "用户性别",
phone LARGEINT COMMENT "用户电话",
address VARCHAR(500) COMMENT "用户地址"

)
ENGINE = olap UNIQUE KEY ( `user_id`,`register_time` )
COMMENT  "This is my first DORIS table"
PARTITION BY RANGE(register_time)
--批量建分区

(
FROM ("2021-01-01") TO ("2030-12-31") INTERVAL 1 MONTH
)
DISTRIBUTED BY HASH ( user_id ) BUCKETS 32
PROPERTIES
(
"function_column.sequence_col" = 'age',
"replication_num" = "1", --副本数。默认副本数为3。如果 BE 节点数量小于3,则需指定副本数小于等于 BE 节点数量。
"enable_unique_key_merge_on_write" = "true",
"in_memory" = "false"
);

4修改doris的ip

image

5.同步数据

image

数据新增成功

image

6.删除数据

image

删除数据成功

image

6.新增刚刚被删除一条一摸一样的数据

image

streamload执行成功

1678692169444

可是数据并没有进入数据库
image

image

image

接下来是很诡异的操作一幕

7.删除数据库,重新建库和建表
参考第2、3点步骤

8.同步数据
image

9.删除数据

需要注意的是,将137行的代码 int age = 30; 改为int age = 28; 【原来是30,现在改为和原始数据一样的数字, 28】
微信图片_20230313161341

数据是可以删除成功的

10.现在同步数据
image

数据可以同步成功
image

11、本人结论

如果 sequence_col 的值和原值一样,删除之后是可以新增成功的,并且可以查询到数据;
如果 sequence_col 的值大于原值,删除之后是可以新增成功的,但是无法查询到数据;

Anything Else?

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Activity

  1. zhannngchen commented on Mar 14, 2023

    @zhannngchen
    Contributor

    discussed with @dearMOMO , the behavior is expected
    sequence column provides a way to get consistent result even the data is loaded out of order
    for more detail, see the document: https://doris.apache.org/zh-CN/docs/dev/data-operate/update-delete/batch-delete-manual

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions