Merge step #2392
Merge step #2392
Conversation
Do just one call to get all the edges for the given nodes, instead of one call for each node. This new call has been added as a new method to Graph and Backends. GetNodesEdges, return the list with all edges for a list of nodes. Batching is used to avoid hitting the max number of clauses set by ES (is set to the default value of 512).
Like Descendants, but using edges in any direction (Descendants only
uses edges from parent to child, Neighbors uses from parent to child and
from child to parent).
The different with the Both step, is Neighbors accumulate nodes seen.
Example (pseudo-syntax):
A -> B -> C
G.V(A).Out().Out() return: C.
But:
G.V(A).Neighbors(2) return: A,B,C
The parameters allowed are the same as in Descendants.
Example:
G.V('foo').Neighbors('RelationType',Within('ownership','foobar'),2)
To improve speed and reduce backend load when using persistent backends,
a new method, GetNodesFromIDs, is implemented in Graph and Backends.
This method only uses one call (or a few in we have hundreds of nodes,
see batching) to get all nodes from the backend.
Batching is used to avoid hitting the max number of clauses set by ES
(is set to the default value of 512).
Node.Copy() creates a copy of the node. It could be useful to return in a query output the modified content of a node without actually modiying it.
|
I am not sure if this step makes sense to anyone else but me :) The full picture is that we are adding/removing events to/from the nodes and we want to be able to see all of those events in the past 24 hours. |
adrianlzt
force-pushed
the
feature/step_merge
branch
2 times, most recently
from
eee82a9
to
b3bbeb8
Compare
|
Thanks ! It does make sense to me too :-) @safchain did something similar when working on the new historical view of the new web UI and added a Valuemap step for this so he may have an opinion on this. Just thinking out loud, I was wondering if we could do something like : |
Aggregates elements from different revisions of the nodes into a new
metadata key.
Given a metadata element, that should be a map[string]interface{},
aggregate different values into another metadata key with format
map[string][]interface{}
Eg.:
Metadata.data V1: {"a":{x}, "b":{y}}
Metadata.data V2: {"a":{z}, "b":{y}}
Metadata.agg: {"a":[{x},{z}], "b":[{y}]}
It's purpose its to show data from past revisions of the same node.
Example:
G.At(1479899809,3600).V().Merge('data','agg')
It could be also called defining the time slice in the parameters
(since, from):
G.V().Merge('A','B',1500000000,1500099999)
This step return a modified copy of the last node, with all the
aggregated data, not the node stored in the graph.
This is to avoid modiying the node stored in the graph.
This PR also modifies the Reduce method of the Neighbors step.
Merge step only needs the node IDs, so Neighbors step could skip
retrieving the full content of nodes.
adrianlzt
force-pushed
the
feature/step_merge
branch
from
b3bbeb8
to
133a80b
Compare
|
@adrianlzt Hello Adrian. Sorry for the delay. I was thinking that a somewhat similar implementation could be to add the Metadata.data V1: {"a":{x}, "b":{y}} G.At(1479899809,3600).V().ValueMap("a, "b").Merge() would return Would it suit your use case ? |
|
Hi @lebauce. More daly here! But maybe you have another better approach. But I always need the full node to draw it (name, type, etc). Does it makes sense? |
Aggregates elements from different revisions of the nodes into a new
metadata key.
Given a metadata element, that should be a map[string]interface{},
aggregate different values into another metadata key with format
map[string][]interface{}
Eg.:
Metadata.data V1: {"a":{x}, "b":{y}}
Metadata.data V2: {"a":{z}, "b":{y}}
Metadata.agg: {"a":[{x},{z}], "b":[{y}]}
It's purpose its to show data from past revisions of the same node.
Example:
G.At(1479899809,3600).V().Merge('data','agg')
It could be also called defining the time slice in the parameters
(since, from):
G.V().Merge('A','B',1500000000,1500099999)
This step return a modified copy of the last node, with all the
aggregated data, not the node stored in the graph.
This is to avoid modiying the node stored in the graph.
This PR also modifies the Reduce method of the Neighbors step.
Merge step only needs the node IDs, so Neighbors step could skip
retrieving the full content of nodes.