Skip to content

No available BE to choose when repair #4104

Closed

Description

@liutang123

Describe the bug
There are A, B, C, D and E 5 BEs.
tablet t has 4 replicas in A(good), B(good), C(version miss) and D(good).

If now mark D and E as decommissioned. FE will never delete replica in C and can not choose D or E to create new replica.

--

The complex situation is:
There are A, B, C, D and E 5 BEs.
tabletA has replicas a1(A)(good), a2(B)(good), a3(C)(version miss), a4(D)(good)
tabletB has replicas b1(A)(good), b2(B)(good), b3(c)(version miss), b4(E)(good)
Because tabletA and tabletB are all REPLICA_RELOCATING, but they can not find a BE to create new replica.
a4 prevents D from being deleted and b4 prevents E from being deleted.
The repair and decommission will hang.

Activity

  1. changed the title [-]No awailable BE to choose when repair[/-] [+]No available BE to choose when repair[/+] on Jul 16, 2020
  2. morningman commented on Jul 20, 2020

    @morningman
    Contributor

    There are A, B, C and D 4 BEs.
    tablet t has 3 replicas in A(good), B(good) and C(version miss).
    If now mark D as decommissioned. FE will never delete replica in C and can not choose D to create new replica.

    I think this case will fall into this branch:

    https://github.com/apache/incubator-doris/blob/2de4f2471bd729e9f723ca8cdd5d9abfe69bf8eb/fe/src/main/java/org/apache/doris/catalog/Tablet.java#L476-L478

    So it can not run to the code you modified in #4105

  3. liutang123 commented on Jul 21, 2020

    @liutang123
    ContributorAuthor

    There are A, B, C and D 4 BEs.
    tablet t has 3 replicas in A(good), B(good) and C(version miss).
    If now mark D as decommissioned. FE will never delete replica in C and can not choose D to create new replica.

    I think this case will fall into this branch:

    https://github.com/apache/incubator-doris/blob/2de4f2471bd729e9f723ca8cdd5d9abfe69bf8eb/fe/src/main/java/org/apache/doris/catalog/Tablet.java#L476-L478

    So it can not run to the code you modified in #4105

    Sorry, decommission one BE may not encounter this bug.
    Please refer to the complicated situation.

    The complex situation is:
    There are A, B, C, D and E 5 BEs.
    tabletA has replicas a1(A)(good), a2(B)(good), a3(C)(version miss), a4(D)(good)
    tabletB has replicas b1(A)(good), b2(B)(good), b3(c)(version miss), b4(E)(good)
    Because tabletA and tabletB are all REPLICA_RELOCATING, but they can not find a BE to create new replica.
    a4 prevents D from being deleted and b4 prevents E from being deleted.
    The repair and decommission will hang.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions