lidongze0629

lidongze0629

Member Since 5 years ago

Alibaba, Beijing China

Experience Points
8
follower
Lessons Completed
4
follow
Lessons Completed
50
stars
Best Reply Awards
42
repos

497 contributions in the last year

Pinned
⚡ 🔨 🍇 💻 🚀 GraphScope: A One-Stop Large-Scale Graph Computing System from Alibaba 来自阿里巴巴的一站式大规模图计算系统 图分析 图查询 图机器学习
⚡ vineyard (v6d): an in-memory immutable data manager. (Project under CNCF)
⚡ 🍇 A C++ library for parallel graph processing (GRAPE) 🍇
⚡ Running MPICH2 on Yarn
Activity
May
20
23 hours ago
Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

Return the original vertex(edge) id during gremlin query

Is your feature request related to a problem? Please describe.

From the results of Gremlin query, it seems that the internal ID is returned, which is very confusing for users because cannot correspond to the original data.

# case 1
>>> client.submit("g.V()").all().result()
[v[0.-9223372036854774811],
 v[0.-9223372036854774810],
 v[0.-9223372036854774809],
 ...]

# case2
>>> client.submit("g.V(1)").all().result()
[v[0.1]]

# case3
>>> client.submit("g.V(1).bothE()").all().result()
[e[-9223372036854756121][0.-9223372036854754957-e0->0.1],
 e[73273][0.1-e0->0.-9223372036854760078],
 e[73276][0.1-e0->0.-9223372036854758343],
 e[73278][0.1-e0->0.-9223372036854758342]]

Describe the solution you'd like Return original vertex(edge) id during gremlin query

lidongze0629
lidongze0629

好的,多谢解答,了解了。

Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

Return the original vertex(edge) id during gremlin query

Is your feature request related to a problem? Please describe.

From the results of Gremlin query, it seems that the internal ID is returned, which is very confusing for users because cannot correspond to the original data.

# case 1
>>> client.submit("g.V()").all().result()
[v[0.-9223372036854774811],
 v[0.-9223372036854774810],
 v[0.-9223372036854774809],
 ...]

# case2
>>> client.submit("g.V(1)").all().result()
[v[0.1]]

# case3
>>> client.submit("g.V(1).bothE()").all().result()
[e[-9223372036854756121][0.-9223372036854754957-e0->0.1],
 e[73273][0.1-e0->0.-9223372036854760078],
 e[73276][0.1-e0->0.-9223372036854758343],
 e[73278][0.1-e0->0.-9223372036854758342]]

Describe the solution you'd like Return original vertex(edge) id during gremlin query

lidongze0629
lidongze0629

@longbinlai Hi, longbin,返回 id 的这里,是不是我们像标准的 TinkerPop Gremlin Server 一样默认返回 属性 id 比较好呢?而不是返回用户不可知的 global unique id

GAIA_IR 的结果如下:

miKrlsY8yMDIyLTA1LTE3IOS4i+WNiDguNTEuNDEucG5n miKrlsY8yMDIyLTA1LTE3IOS4i+WNiDkuMDcuMDMucG5n
May
19
1 day ago
Activity icon
issue

lidongze0629 issue alibaba/GraphScope

lidongze0629
lidongze0629

Support `valueMap` operation in GAIA_IR

Is your feature request related to a problem? Please describe.

print(interactive.execute('g.V().has("id", "Domain_1").bothE().valueMap()').all())

File /work/python/graphscope/client/rpc.py:193, in GRPCClient._run_step_impl(self, runstep_requests)
    187     logger.error(
    188         "Runstep failed with code: %s, message: %s",
    189         error_codes_pb2.Code.Name(response.code),
    190         response.error_msg,
    191     )
    192     if response.full_exception:
--> 193         raise pickle.loads(response.full_exception)
    194 return response

RuntimeError: Fetch gremlin result failed

GIE frontend log 截屏2022-05-19 下午4 19 13

Activity icon
issue

lidongze0629 issue alibaba/GraphScope

lidongze0629
lidongze0629

Package graphscope wheel(alpha) with GAIA

as titled.

Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

Package graphscope wheel(alpha) with GAIA

as titled.

lidongze0629
lidongze0629

Close as GAIA IR already packaged in the wheel, enable by environment USE_GAIA_ENGINE=true

May
18
2 days ago
Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

[BUG] g.graphlearn()/g.gremlin() should check if the type is compatible first

Describe the bug

GraphLearn has implies int64_t as OID_T (see types.h) and current gremlin engine doesn't support string oids as well (see https://github.com/alibaba/GraphScope/issues/1265).

An exceptions should be thrown in g.graphlearn()/g.gremlin() is the oid type of vineyard graph is not compatible with those two engines.

lidongze0629
lidongze0629

sess.gremlin failed with string oid

WARNING: Logging before InitGoogleLogging() is written to STDERR
I0519 01:01:55.693738 61686 htap_ds_impl.cc:35] Initialize vineyard client
I0519 01:01:55.784637 61686 htap_ds_impl.cc:39] Get vineyard object ok: 0x565099020260
I0519 01:01:55.784672 61686 htap_ds_impl.cc:43] FragGroup: total frag num = 2, vertex label num = 8, edge label num = 11
I0519 01:01:55.785598 61686 htap_ds_impl.cc:82] fid = 0, instance_id = 0, location = 0
[error] Assertion failed in "meta.GetTypeName() == __type_name": , in function void vineyard::ArrowFragment<OID_T, VID_T>::Construct(const vineyard::ObjectMeta&) [with OID_T = long int; VID_T = long unsigned int], file /usr/local/lib/cmake/vineyard/../../../include/vineyard/graph/fragment/arrow_fragment.vineyard.h, line 136Expect typename 'vineyard::ArrowFragment<int64,uint64>', but got 'vineyard::ArrowFragment<std::string,uint64>'
fatal runtime error: Rust cannot catch foreign exceptions
May
17
3 days ago
Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

2022-05-16 16:48:28,480 [ERROR][rpc:188]: Runstep failed with code: INTERACTIVE_ENGINE_INTERNAL_ERROR, message: Error occurred during preprocessing, The traceback is: Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 858, in _create_interactive_instance if check_gremlin_server_ready(maxgraph_endpoint): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/utils.py", line 1933, in check_gremlin_server_ready raise TimeoutError(f"Gremlin check query failed: {error_message}") TimeoutError: Gremlin check query failed: ws:///gremlin

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 624, in _RunStep head, bodies = self.run_on_interactive_engine(dag) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 514, in run_on_interactive_engine op_result = self._create_interactive_instance(op) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 884, in _create_interactive_instance raise RuntimeError("Create interactive instance failed.") from e RuntimeError: Create interactive instance failed.

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1154, in gremlin _wrapper = self._wrapper( File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 936, in _wrapper return self.run(dag_node) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 964, in run response = self._grpc_client.run(fetch_handler.targets) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 102, in run return self._run_step_impl(runstep_requests) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 155, in with_grpc_catch return fn(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 194, in _run_step_impl raise pickle.loads(response.full_exception) RuntimeError: Create interactive instance failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test.py", line 43, in interactive = graphscope.gremlin(graph) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1533, in gremlin return get_default_session().gremlin(graph, engine_params) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 357, in wrapper return_value = func(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1161, in gremlin raise InteractiveEngineInternalError(str(e)) from e graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

lidongze0629
lidongze0629

@caoxiang93 目前的日志还看不出什么问题,方便的话,可以加一下我们的微信哈,发一份全的日志在详细看看,微信号: graphscope

Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

2022-05-16 16:48:28,480 [ERROR][rpc:188]: Runstep failed with code: INTERACTIVE_ENGINE_INTERNAL_ERROR, message: Error occurred during preprocessing, The traceback is: Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 858, in _create_interactive_instance if check_gremlin_server_ready(maxgraph_endpoint): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/utils.py", line 1933, in check_gremlin_server_ready raise TimeoutError(f"Gremlin check query failed: {error_message}") TimeoutError: Gremlin check query failed: ws:///gremlin

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 624, in _RunStep head, bodies = self.run_on_interactive_engine(dag) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 514, in run_on_interactive_engine op_result = self._create_interactive_instance(op) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 884, in _create_interactive_instance raise RuntimeError("Create interactive instance failed.") from e RuntimeError: Create interactive instance failed.

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1154, in gremlin _wrapper = self._wrapper( File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 936, in _wrapper return self.run(dag_node) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 964, in run response = self._grpc_client.run(fetch_handler.targets) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 102, in run return self._run_step_impl(runstep_requests) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 155, in with_grpc_catch return fn(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 194, in _run_step_impl raise pickle.loads(response.full_exception) RuntimeError: Create interactive instance failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test.py", line 43, in interactive = graphscope.gremlin(graph) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1533, in gremlin return get_default_session().gremlin(graph, engine_params) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 357, in wrapper return_value = func(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1161, in gremlin raise InteractiveEngineInternalError(str(e)) from e graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

lidongze0629
lidongze0629

executor.log 看样子没问题,maxgraph-coordinator.logmaxgraph-coordinator.log 的日志可以贴一下嘛

Activity icon
issue

lidongze0629 issue alibaba/GraphScope

lidongze0629
lidongze0629

Specify the port of the interactive engine from the coordiantor

Is your feature request related to a problem? Please describe. Currently, the coordinator gets the GIE service address from the log maxgraph-frontend.log, which may cause the engine failed to start by the system environment limit such as permission denied.

Describe the solution you'd like It would be great to let the coordinator specify an available port for GIE service.

Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

2022-05-16 16:48:28,480 [ERROR][rpc:188]: Runstep failed with code: INTERACTIVE_ENGINE_INTERNAL_ERROR, message: Error occurred during preprocessing, The traceback is: Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 858, in _create_interactive_instance if check_gremlin_server_ready(maxgraph_endpoint): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/utils.py", line 1933, in check_gremlin_server_ready raise TimeoutError(f"Gremlin check query failed: {error_message}") TimeoutError: Gremlin check query failed: ws:///gremlin

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 624, in _RunStep head, bodies = self.run_on_interactive_engine(dag) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 514, in run_on_interactive_engine op_result = self._create_interactive_instance(op) File "/usr/local/python/lib/python3.8/site-packages/gscoordinator/coordinator.py", line 884, in _create_interactive_instance raise RuntimeError("Create interactive instance failed.") from e RuntimeError: Create interactive instance failed.

Traceback (most recent call last): File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1154, in gremlin _wrapper = self._wrapper( File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 936, in _wrapper return self.run(dag_node) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 964, in run response = self._grpc_client.run(fetch_handler.targets) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 102, in run return self._run_step_impl(runstep_requests) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 155, in with_grpc_catch return fn(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/rpc.py", line 194, in _run_step_impl raise pickle.loads(response.full_exception) RuntimeError: Create interactive instance failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "test.py", line 43, in interactive = graphscope.gremlin(graph) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1533, in gremlin return get_default_session().gremlin(graph, engine_params) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/utils.py", line 357, in wrapper return_value = func(*args, **kwargs) File "/usr/local/python/lib/python3.8/site-packages/graphscope/client/session.py", line 1161, in gremlin raise InteractiveEngineInternalError(str(e)) from e graphscope.framework.errors.InteractiveEngineInternalError: Create interactive instance failed.

lidongze0629
lidongze0629

@caoxiang93 很有可能是写日志的权限原因导致的,本地拉起方式下,日志默认会写到 /var/log/graphscope 下,如果没有权限,则尝试写到 ${HOME}/.local/log/graphscope 下,你可以检查下这两个路径,大致对应 GIE 的日志结构如下:

...
-rw-rw-r-- 1 graphscope graphscope  executor.log
-rw-rw-r-- 1 graphscope graphscope  maxgraph-coordinator.log
-rw-rw-r-- 1 graphscope graphscope  maxgraph-frontend.log

如果正常生成了日志,还是有 Create interactive instance failed 的错误,则可以将日志内的报错信息贴出来哈

May
14
6 days ago
push

lidongze0629 push GraphScope/cpp-template

lidongze0629
lidongze0629

find graphscope module from per-user site-packages

lidongze0629
lidongze0629

commit sha: 4afee74f98e003059ef26a6dc56db3b89b1c162a

push time in 6 days ago
May
12
1 week ago
May
11
1 week ago
Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

Providing Java/C++ SDK in github `public template`

a template project for users to develop apps with SDK

  • C++
  • Java
May
10
1 week ago
Activity icon
issue

lidongze0629 issue comment alibaba/GraphScope

lidongze0629
lidongze0629

能否提供阿里云所有docker镜像的列表以及每个docker镜像的描述?

1.能否在文档中增加对docker镜像的描述? 2.如何能够访问到阿里云的私有docker仓库? 3. 当前是否提供了支持pytorch的graphscope镜像的版本?

lidongze0629
lidongze0629

感谢您的回答。 初学docker不久,请教一下registry.cn-hongkong.aliyuncs.com/graphscope应该是你们搭建的docker私有仓库吧?尝试过docker pull,可以拉取对应tag的镜像,但是暂时没有找到查询registry.cn-hongkong.aliyuncs.com/graphscope目录下所有镜像列表的方式。

是的,目前还无法查询 registry.cn-hongkong.aliyuncs.com/graphscope 下的所有 tag,我们计划会在 dockerhub 上同步一份,现阶段每个 GraphScope Release 都会对应一个 tag,如 registry.cn-hongkong.aliyuncs.com/graphscope:0.13.0

push

lidongze0629 push alibaba/GraphScope

lidongze0629
lidongze0629

Add release note 0.13.0. (#1538)

  • Create release-notes-0.13.0.html

commit sha: 657fe96b04fdc6969007fd181af52c914e461482

push time in 1 week ago
pull request

lidongze0629 pull request alibaba/GraphScope

lidongze0629
lidongze0629

Add release note 0.13.0.

What do these changes do?

Related issue number

Fixes

May
9
1 week ago
Activity icon
issue

lidongze0629 issue alibaba/GraphScope

lidongze0629
lidongze0629

code linter for `interactive_engine`

A unified code style makes less code diff lines and have many benefits.

To be consistent with the C++ part in the repo, which follows google style. For Java codes, we also choose Google Java code style.

Here is a tool google-java-format makes code formatting easy. For developers using IDE, follow this to configurate.

When formatting code, Please remeber to choose AOSP standand which is google style with 4-spaces indentation compliance.

From Nov., the style checker will be added to CI. PR failed to pass the style check will NOT be merged.

Activity icon
created branch
createdAt 1 week ago
Activity icon
issue

lidongze0629 issue alibaba/GraphScope

lidongze0629
lidongze0629

Refactor the giectl script and implement the scheduling processes in the coordinator

Is your feature request related to a problem? Please describe. As the title says, It will help the coordinator manage the lifecycle of GIE better, and can also make the coordinator manage the log information more effectively.

Additional context Add any other context or screenshots about the feature request here.

Previous