neo4j 数据库使用 Bolt 转换 RestApi 请求结果格式

tsvico Lv5
  2020-12-22 22:27:26 2020-12-22 22:27:26 创建   2021-03-06 20:21:07 2021-03-06 20:21:07 更新   1.3k 字  6 分钟  

在做图数据库的数据库可视化的时候，使用到了 eisman/neo4jd3这个可视化库，查看了一下其数据格式，发现和 neo4j 自带的 RestApi 接口大致一致

d3 库数据格式为👇，

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
{
    "results": [
        {
            "columns": ["user", "entity"],
            "data": [
                {
                    "graph": {
                        "nodes": [
                            {
                                "id": "1",
                                "labels": ["User"],
                                "properties": {
                                    "userId": "eisman"
                                }
                            },
                            {
                                "id": "8",
                                "labels": ["Project"],
                                "properties": {
                                    "name": "neo4jd3",
                                    "title": "neo4jd3.js",
                                    "description": "Neo4j graph visualization using D3.js.",
                                    "url": "https://eisman.github.io/neo4jd3"
                                }
                            }
                        ],
                        "relationships": [
                            {
                                "id": "7",
                                "type": "DEVELOPES",
                                "startNode": "1",
                                "endNode": "8",
                                "properties": {
                                    "from": 1470002400000
                                }
                            }
                        ]
                    }
                }
            ]
        }
    ],
    "errors": []
}

而 neo4j RestAPI 返回的数据为↓，可以看到两者的格式还是有些地方不一样的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
{
    "results": [
        {
            "columns": [
                "p"
            ],
            "data": [
                {
                    "row": [
                        [
                            {
                                "updateDate": 1608600619,
                                "departmentId": 880,
                                "sex": "35",
                                "mobile": "11112247826",
                                "classificationLevel": "非密",
                                "political": "党员",
                                "birth": 801936000,
                                "userCode": "110000200001010001",
                                "syncId": "1",
                                "post": "会计",
                                "isImportant": false,
                                "leaveDate": 1608600622,
                                "name": "张三",
                                "national": "",
                                "email": "",
                                "education_degree": "",
                                "createDate": 1607576509,
                                "status": "在职"
                            },
                            {},
                            {
                                "updateDate": 1608600619,
                                "name": "财务部",
                                "pid": 879,
                                "order": 4,
                                "syncId": "880"
                            }
                        ]
                    ],
                    "meta": [
                        [
                            {
                                "id": 22,
                                "type": "node",
                                "deleted": false
                            },
                            {
                                "id": 2,
                                "type": "relationship",
                                "deleted": false
                            },
                            {
                                "id": 1,
                                "type": "node",
                                "deleted": false
                            }
                        ]
                    ]
                }
            ]
        }
    ],
    "errors": []
}

使用 postman 请求 Neo4j REST API 文档

  • 这里用到的接口 /db/data/transaction/commit
  • POST body
1
2
3
4
5
{
  "statements" : [{
      "statement": "MATCH p=(n:oa_userInfo)-[:`部门`]-() where n.name='张三' RETURN p LIMIT 25"
    }]
}

img

这里要加上鉴权，以及请求头 Content-Type: application/json
在这里插入图片描述
这样就可以得到上边的结果，可以使用 neo4jd3 对数据渲染

由于在系统中已经使用了 bolt 的连接方式，于是尝试使用 neo4j.ogm 驱动中的 session 去请求查询，将结果转换，返回和 RestApi 一样的结果
在 springboot 中可以直接取得 SessionFactory (编码使用的是 kotlin)

1
2
@Autowired
 private lateinit var sessionFactory: SessionFactory

声明一个结果类

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
class QueryRest {

    data class results(
            var errors: MutableList<Any>?,
            var results: MutableList<Result>?
    )

    data class Result(
            var columns: MutableList<String>,
            var `data`: MutableList<Data>
    )

    data class Data(
            var graph: Graph
    )

    data class Graph(
            var nodes: List<Node>,
            var relationships: List<Relationship>
    )

    data class Node(
            var id: String,
            var labels: ArrayList<String>,
            var properties: Any?
    )

    data class Relationship(
            var endNode: String,
            var id: String,
            var properties: Any?,
            var startNode: String,
            var type: String
    )
}

ServerImpl

初步实现的效果，直接传递 Cypher 语句并不严谨，可能会存在注入问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
override fun query(sql: String): QueryRest.results {
    val session: Session = sessionFactory.openSession()
    val grep = session.query(sql, mutableMapOf<String, String>())
    val results = mutableListOf<QueryRest.Result>()
    val columns = mutableListOf<String>()

    val datas = mutableListOf<QueryRest.Data>()
    grep.forEach { map ->
        map.forEach { (key, value) ->
            columns.add(key)
            val v = value as Array<*>
            v.forEach {
                val vv = it as Path.Segment
                val nodeStart = QueryRest.Node(
                        id = vv.start().id().toString(),
                        labels = vv.start().labels() as ArrayList<String>,
                        properties = vv.start().asMap()
                )
                val nodeEnd = QueryRest.Node(
                        id = vv.end().id().toString(),
                        labels = vv.end().labels() as ArrayList<String>,
                        properties = vv.end().asMap()
                )
                val nodes = mutableListOf<QueryRest.Node>(nodeStart, nodeEnd)
                val relationship = QueryRest.Relationship(
                        id = vv.relationship().id().toString(),
                        type = vv.relationship().type(),
                        startNode = vv.relationship().startNodeId().toString(),
                        endNode = vv.relationship().endNodeId().toString(),
                        properties = vv.relationship().asMap()
                )
                val relationships = mutableListOf<QueryRest.Relationship>(relationship)
                val graph = QueryRest.Graph(nodes, relationships)
                datas.add(QueryRest.Data(graph))
                logger.info("$value -- $vv")
            }
        }
    }
    results.add(QueryRest.Result(columns.distinct() as MutableList<String>, datas))
    return QueryRest.results(mutableListOf(), results)
}

over

2020/12/22 更新

使用过程中发现 ogm 的一个 TODO，是否自动映射 Bean，现在我使用的 neo4j-ogm-bolt-driver:3.2.18 默认是开启映射的，所以上方的转换代码可能会出现一些问题，影响到我们功能的使用。
Are we going to use the neo4jOperations conversion method to cast the value object to its proper class?

1
2
3
4
5
6
7
8
9
10
11
12
/**
 * a cypher statement this method will return a Result object containing a collection of Map's which represent Neo4j
 * objects as properties, along with query statistics if applicable.
 * Each element of the query result is a map which you can access by the name of the returned field
 * TODO: Are we going to use the neo4jOperations conversion method to cast the value object to its proper class?
 *
 * @param cypher     The parameterisable cypher to execute.
 * @param parameters Any parameters to attach to the cypher.
 * @param readOnly   true if the query is readOnly, false otherwise
 * @return A {@link Result} of {@link Iterable}s with each entry representing a neo4j object's properties.
 */
Result query(String cypher, Map<String, ?> parameters, boolean readOnly);

在这里插入图片描述
通过 Debug neo4j-ogm 源代码，找到自动转换的类，发现无法使用重写解决，经过研究 Neo4jSession 的类变量，针对这个问题，对原有的 handlers 做了如下更改：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
private fun handlers(grep: org.neo4j.ogm.model.Result): QueryRest.results {
        val results = mutableListOf<QueryRest.Result>()
        val columns = mutableListOf<String>()
        val daTas = mutableListOf<QueryRest.Data>()
        grep.forEach { map ->
            map.forEach { (key, value) ->
                columns.add(key)
                if (value is Array<*>) {
                    // 是查询的结点关系数据
                    // MATCH p=(n)-[r]-()
                    value.forEach {
                        val vv = it as Path.Segment
                        val nodeStart = QueryRest.Node(
                            id = vv.start().id().toString(),
                            labels = vv.start().labels() as ArrayList<String>,
                            properties = vv.start().asMap()
                        )
                        val nodeEnd = QueryRest.Node(
                            id = vv.end().id().toString(),
                            labels = vv.end().labels() as ArrayList<String>,
                            properties = vv.end().asMap()
                        )
                        val nodes = mutableListOf<QueryRest.Node>(nodeStart, nodeEnd)
                        val relationship = QueryRest.Relationship(
                            id = vv.relationship().id().toString(),
                            type = vv.relationship().type(),
                            startNode = vv.relationship().startNodeId().toString(),
                            endNode = vv.relationship().endNodeId().toString(),
                            properties = vv.relationship().asMap()
                        )
                        val relationships = mutableListOf<QueryRest.Relationship>(relationship)
                        val graph = QueryRest.Graph(nodes, relationships)
                        daTas.add(QueryRest.Data(graph))
                    }
                } else if (value is NodeModel) {
                    // 是查询的结点数据
                    // MATCH (n:make_sec_entity) RETURN n LIMIT 3
                    val arraylist = ArrayList<String>(7)
                    value.labels.forEach {
                        arraylist.add(it)
                    }
                    val propertiesMap = HashMap<String, String>(8)
                    value.propertyList.forEach {
                        propertiesMap[it.key] = it.value.toString()
                    }
                    val node = QueryRest.Node(
                        id = value.id.toString(),
                        labels = arraylist,
                        properties = propertiesMap
                    )
                    val nodes = mutableListOf(node)
                    daTas.add(QueryRest.Data(QueryRest.Graph(nodes, null)))
                    // throw BizException(500, "暂时不可查询未计算关系的数据")
                } else {
                    val node = beanToNodeModel(value)
                    val nodes = mutableListOf(node)
                    daTas.add(QueryRest.Data(QueryRest.Graph(nodes, null)))
                }
            }
        }
        results.add(QueryRest.Result(columns.distinct() as? MutableList<String>, daTas))
        if (results.size != 0) {
            logger.info("-*---图数据查询成功")
        }
        return QueryRest.results(mutableListOf(), results)
    }

    private fun beanToNodeModel(bean: Any): QueryRest.Node {
        val neo4jSession = session as Neo4jSession
        val metadata = neo4jSession.metaData()
        val label = metadata.classInfo(bean::class.java).neo4jName()
        val id = metadata.classInfo(bean::class.java).getFieldInfo("id").read(bean)
        return QueryRest.Node(
            id = id.toString(),
            labels = arrayListOf(label),
            properties = bean
        )
    }
  • 标题: neo4j 数据库使用 Bolt 转换 RestApi 请求结果格式
  • 作者: tsvico
  • 创建于 : 2020-12-22 22:27:26
  • 更新于 : 2021-03-06 20:21:07
  • 链接: https://blog.tbox.fun/2020/e526b597.html
  • 版权声明: 本文章采用 CC BY-NC-SA 4.0 进行许可。
评论
目录
neo4j 数据库使用 Bolt 转换 RestApi 请求结果格式