I meet a problem.

The computer has alink 1.4 and pyflink 1.13 and flink 1.10.

When I run alink programs, it runs into a classnotfound exception. In my view, it is a jar confliction. The case is as below. it won't call the jars in pyflink. The job would be executed in flink 1.10(it is launched beforehand）. So the classpath or memory somewhere has flink1.10, and alink choose flink1.10, not pyflink1.13.

I wanna know how the alink make the decision about which flink to choose, and what is the procedure?

I can't call fit() on pipeline created by Pipeline.load("pipeline.ak") in python, because return object doesn't have fit() method . But this method works in Java.

Reason

In Java, Pipeline.load() return a Pipeline object. In Python, it returns a PipelineModel object. One misspelled calss name in pyalink/alink/pipeline/base.py Pipeline.load() appears to cause this.

class Pipeline(Estimator):
    ......
    @staticmethod
    def collectLoad(operator: BatchOperator):
        _j_pipeline_cls = get_java_class("com.alibaba.alink.pipeline.Pipeline")
        j_pipeline = _j_pipeline_cls.collectLoad(operator.get_j_obj())
        stages = Pipeline._check_lazy_params(j_pipeline)
        return Pipeline(*stages, j_pipeline=j_pipeline)

    @staticmethod
    def load(file_path: Union[str, FilePath]):
        _j_pipeline_cls = get_java_class("com.alibaba.alink.pipeline.PipelineModel")
        j_pipeline = _j_pipeline_cls()
        if isinstance(file_path, (str,)):
            path = file_path
            j_pipeline = _j_pipeline_cls.load(path)
        elif isinstance(file_path, FilePath):
            operator = file_path
            j_pipeline = _j_pipeline_cls.load(operator.get_j_obj())
        else:
            raise ValueError("file_path must be str or FilePath")

        stages = Pipeline._check_lazy_params(j_pipeline)
        return Pipeline(*stages, j_pipeline=j_pipeline)

Solution

use collectLoad() instead of load() in pyalink.

As the title says, BatchOperator#firstN will cause the task stuck when using Flink >= 1.13.1.

Possible reason： BatchOperator#firstNcallsDataSet#first, then calls FirstReducer#reduce. But FirstReducer#reduce` does not exhaust the values iterable.

Possible workaround:

1. Use Flink <= 1.13.0;
2. Do not useBatchOperator#firstN`.

Support libtorch model inference through Alink.

pyalink代码在alink中移除了吗，怎么没有找到？

使用git 克隆代码会产生一个error: git clone https://github.com/alibaba/Alink.git Cloning into 'Alink'... remote: Enumerating objects: 19652, done. remote: Counting objects: 100% (13723/13723), done. remote: Compressing objects: 100% (7412/7412), done. remote: Total 19652 (delta 6743), reused 11736 (delta 5290), pack-reused 5929Receiving objects: 100% (19652/19652), 11.29 MiB | 1.75 MiB/s Receiving objects: 100% (19652/19652), 11.48 MiB | 1.65 MiB/s, done. Resolving deltas: 100% (9599/9599), done. error: invalid path 'docs/cn/批组件/特征工程/特征构造: OverWindow (OverWindowBatchOp).md' fatal: unable to checkout working tree warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

win10 系统 git clone 之分支代码 报错 []( ) 出错文件如下： docs/cn/批组件/特征工程/特征构造

Support libtorch model inference through Alink.