Skip to content

ARTEMIS-6144 Improve getPagedResult - #6563

Open
GChuf wants to merge 4 commits into
apache:mainfrom
GChuf:ARTEMIS-6144
Open

GChuf wants to merge 4 commits into
apache:mainfrom
GChuf:ARTEMIS-6144

Conversation

@GChuf

@GChuf GChuf commented Jul 6, 2026 •

Copy link
Copy Markdown
Contributor

Benchmarks below:

Benchmark addressCount Throughput (main) Throughput (6144) Δ Throughput Alloc (main, B/op) Alloc (6144, B/op) Δ Alloc
testAllResults 1000 1,127.29 ops/s 3,583.19 ops/s +217.9% 37,502.19 47,313.95 +26.2%
testAllResults 10000 82.95 ops/s 219.52 ops/s +164.6% 449,604.60 460,263.58 +2.4%
testFirstPage 1000 1,126.65 ops/s 3,661.16 ops/s +225.0% 22,958.20 47,289.91 +106.0%
testFirstPage 10000 80.90 ops/s 229.92 ops/s +184.2% 280,597.19 460,238.15 +64.0%
testMiddlePage 1000 1,127.60 ops/s 3,591.73 ops/s +218.5% 23,422.20 47,529.95 +102.9%
testMiddlePage 10000 81.81 ops/s 219.18 ops/s +167.9% 281,651.85 460,479.60 +63.5%

@GChuf
GChuf marked this pull request as draft July 6, 2026 13:19
@GChuf
GChuf marked this pull request as ready for review July 7, 2026 12:04
@GChuf

GChuf commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Ready for review.
After a bunch of sanity checks:
The JMH tests show a positive improvement in speed, a bit higher memory usage though - but the mem usage is small.
JProfiler shows reduction in relative cpu usage from 20-25% to about 6-10%.
The majority of the benefits, and all the extra mem usage come from fieldCache = new IdentityHashMap<>(array.length). Otherwise I was getting better memory numbers before I added the IdentityHashMap.

Map<Object, Object> fieldCache = new IdentityHashMap<>(array.length);
for (T item : array) {
if (item != null) {
fieldCache.put(item, getField(item, sortField));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the old code, getField() was called inside the comparator's try/catch, so an exception during field extraction was swallowed and treated as equal. However, the new code computes fields in a pre-loop with no exception handling.

This could be a problem for views like QueueView where getField() does server.locateQueue(...) and dereferences the result. Here if a queue is deleted mid-listing you get an NPE. Previously one bad element sorted as "equal" and the listing succeeded; now a single bad element throws out of getPagedResult, failing the entire management query.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - added a try/catch in the loop.

this.sortOrder = json.getString(SORT_ORDER);
} else if (json.containsKey(SORT_COLUMN)) {
this.sortField = json.getString(SORT_COLUMN);
this.sortOrder = json.getString(SORT_ORDER);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A payload with SORT_COLUMN but no SORT_ORDER now throws where the old code didn't.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed by adding a check if json contains SORT_ORDER.

Arrays.sort(array, cachedComparator);

if (page == -1 || pageSize == -1) {
return Arrays.asList(array);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This now returns a mutable Arrays.asList(...) instead of Collections.unmodifiableList(...). This might not make any difference, but it's a contract change.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code now returns Collections.unmodifiableList(collectionList) or Collections.unmodifiableList(collectionList.subList(start, end)).

Comment on lines +139 to +146
if (sortOrderDescending) {
if (rightValue instanceof Comparable r) {
return r.compareTo(leftValue);
}
} else {
if (leftValue instanceof Comparable l) {
return l.compareTo(rightValue);
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getField returns null for absent/optional fields, so a sort field with some null values is realistic.

Consider the scenario with ascending order, leftValue = abc, and rightValue = null:

  • Old code: rightValue instanceof Comparable is false so return 0. Clean, no exception.
  • New code: only leftValue is checked; it is Comparable, so it calls "abc".compareTo(null) throwing a NullPointerException which is caught so return 0.

The result (i.e., 0) is the same, but the new path throws and catches an exception on roughly every comparison that involves a null, inside the hottest loop of the sort. This is orders of magnitude more expensive than the avoided instanceof. So sorting any data containing a null will be slower, not faster.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new code now also checks for null values before executing the compareTo method:

 if (leftValue == rightValue) {
    return 0;
 }
 // push nulls to bottom of the list
 if (leftValue == null) {
    return 1;
 }
 if (rightValue == null) {
    return -1;
 }

@GChuf
GChuf marked this pull request as draft September 3, 2026 06:23
@GChuf
GChuf marked this pull request as ready for review September 4, 2026 10:27
GChuf and others added 2 commits October 26, 2026 12:54
Sort values can be null, e.g. the client ID of a connection without
one. A comparator which treats null as equal to everything violates
the Comparator contract, so sorting failed with "Comparison method
violates its general contract!". Nulls must go to the bottom of the
list in both sort orders.

TimSort only detects contract violations with 32 or more elements and
only for some orderings, so the test sorts 20 shuffled orderings of 64
elements before checking any result.

Assisted-by: Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The values of one sort field can have different types. InVM
connection IDs are Strings while Netty connection IDs are ChannelIds,
so sorting connections by connectionID (the default for
listConnections) compares a String with a ChannelId when both kinds of
connections exist. Calling compareTo across such types throws
ClassCastException, and treating them as equal violates the
Comparator contract.

testSortWithMixedValueTypes sorts 64 values mixing nulls, Integers,
Strings and values which are not Comparable. Like the null value test
it sorts 20 shuffled orderings in both directions before checking any
result, then checks that nulls are last and that the other values are
grouped by type and ordered within each type.

Assisted-by: Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@GChuf

GChuf commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

The comparator was changed - another function was added: compareValues.
Both classes are now checked - if the values' classes which are compared are not the same, then the values are sorted based on their class' name first. If both left and right value are Comparable, they're compared the same way as before. If not, they're both converted to string first.

According to claude, comparison between 2 different classes could happen, for example, when comparing ConnectionView - CONNECTION_ID (String vs io.netty.channel.DefaultChannelId):

InVM connections use a UUID String; every Netty connection (CORE, AMQP, MQTT, STOMP, OpenWire) uses a Netty ChannelId. ManagementRemotingConnection also uses a String ("management").

This PR is now also connected to ARTEMIS-6321 - which was already solved here with the null checks before the latest push.

Regarding the performance, I'm getting wildly different mem per op numbers for main(8e2d6ab) now.
Currently the numbers look like these changes are a win/win situation performance wise.

Benchmark addressCount Throughput (main) Throughput (6144) Δ Throughput Alloc (main, B/op) Alloc (6144, B/op) Δ Alloc
testAllResults 1000 1,121.17 ops/s 2,937.20 ops/s +162.0% 449,534.27 47,338.37 -89.5%
testAllResults 10000 80.64 ops/s 190.39 ops/s +136.1% 6,152,229.83 460,292.31 -92.5%
testFirstPage 1000 1,106.16 ops/s 2,989.91 ops/s +170.3% 434,158.33 47,370.34 -89.1%
testFirstPage 10000 81.03 ops/s 187.24 ops/s +131.1% 5,984,757.50 460,325.01 -92.3%
testMiddlePage 1000 1,133.40 ops/s 2,954.94 ops/s +160.7% 434,590.19 47,370.37 -89.1%
testMiddlePage 10000 79.83 ops/s 189.71 ops/s +137.7% 5,979,503.06 460,324.58 -92.3%

Sorry if this became a bit of a mess.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants