ptitzler

ptitzler

Member Since 6 years ago

Center for Open-Source Data & AI Technologies, San Jose, CA, USA

Experience Points
35
follower
Lessons Completed
2
follow
Lessons Completed
38
stars
Best Reply Awards
84
repos

962 contributions in the last year

Pinned
⚡ Information and links you might find useful. Or not.
Activity
Oct
13
3 days ago
pull request

ptitzler pull request elyra-ai/examples

ptitzler
ptitzler

Improve component makefiles

  • Add new targets for publication on quay.io and hub.docker.com
  • Fix variable typo related to hub.docker.com processing

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
Activity icon
created branch

ptitzler in ptitzler/examples create branch improve-component-makefile

createdAt 2 days ago
push

ptitzler push ptitzler/elyra

ptitzler
ptitzler

Update changelog for release 3.2.0

ptitzler
ptitzler

Prepare for next development iteration

ptitzler
ptitzler

Add front end integration tests for python and R editor operations (#2216)

commit sha: 3f9e320d581cf986a3c146071595eff7bc4d2bd7

push time in 2 days ago
push

ptitzler push ptitzler/examples

ptitzler
ptitzler

Add link to custom KFP component tutorial to repository readme (#74)

ptitzler
ptitzler

Fix markdown formatting issue (#73)

ptitzler
ptitzler

Update kubeflow pipelines component examples (#71)

Include download component in example pipelines to seed downstream components that require data via input paths

Fixes elyra-ai/elyra#2024 Fixes elyra-ai/elyra#2025

ptitzler
ptitzler

Add truncate-file and split-file KFP example custom components (#75)

  • Add truncate-file component

  • Fix readme

  • Add split file example component

  • Move component spec to proper location

  • Fix container image reference in component specification

  • Fix container image references again

  • Fix container image bug

commit sha: 7a5cd5b4615812f0044c25e9992f81d7df4d59c9

push time in 2 days ago
Activity icon
delete

ptitzler in elyra-ai/examples delete branch add-kfp-components

deleted time in 2 days ago
pull request

ptitzler pull request elyra-ai/examples

ptitzler
ptitzler

Add truncate-file and split-file KFP example custom components

This PR

  • adds two new example KFP custom components
  • Fixes issues in other example KFP custom component READMEs

Example pipeline that utilizes both components:

image

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
push

ptitzler push elyra-ai/examples

ptitzler
ptitzler

Add truncate-file and split-file KFP example custom components (#75)

  • Add truncate-file component

  • Fix readme

  • Add split file example component

  • Move component spec to proper location

  • Fix container image reference in component specification

  • Fix container image references again

  • Fix container image bug

commit sha: 7a5cd5b4615812f0044c25e9992f81d7df4d59c9

push time in 2 days ago
Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Support KFP with LDAP Dex auth

What changes were proposed in this pull request?

This PR resolves https://github.com/elyra-ai/elyra/issues/2210

  • support LDAP Dex authentication for KFP
  • if both LDAP and staticPasswords authentication are enabled, we use staticPasswords

How was this pull request tested?

This PR was tested locally in the following configurations:

  • staticPasswords enabled only
  • LDAP enabled only
  • staticPasswords and LDAP enabled

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

The flow you are describing is as expected, and is exactly what this PR handles.

Let me clarify inline.

If multiple providers are enabled:

That's our deployment (static + ldap).

  1. GET the URL: https://kubeflow.example.com

    • Be redirected to https://kubeflow.example.com/auth?=xxxxx

Actual redirect URL received from DEX:

http://scrip1.fyre.ibm.com:32402/dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=MT...K0G
  1. Rewrite URL with /auth/local path

Actual rewrite result:

http://scrip1.fyre.ibm.com:32402/dex/auth/local?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc& ...
  1. GET the [rewritten] URL

HTTP 500 from KFP/DEX

I should note that interactively (in a browser) the setup works just fine because the user selects a provider on the initial DEX page, which redirects to the proper authentication URL (e.g. http://scrip1.fyre.ibm.com:32402/dex/auth/ldap?req=kvkqnxrira4dxscaxwx66lntl

Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Error running pipeline with custom components

Created a pipeline with "run notebook using papermill" kfp custom component. Notebook parameter of the component is updated with notebook https://github.com/elyra-ai/examples/blob/master/pipelines/kubeflow_pipelines_component_examples/run_notebook_component/a-notebook.ipynb.

Below is error message while running the pipeline.

Error Message : Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 207, in process kfp_argo_compiler.Compiler().compile(pipeline_function, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1179, in compile self._create_and_write_workflow( File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1231, in _create_and_write_workflow workflow = self._create_workflow(pipeline_func, pipeline_name, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1009, in _create_workflow pipeline_func(*args_list, **kwargs_dict) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 194, in pipeline_function = lambda: self._cc_pipeline(pipeline, #8 E731 File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 534, in _cc_pipeline self.log.debug(f"Processing component parameter '{component_property.name}' " AttributeError: 'ComponentParameter' object has no attribute 'type'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute result = await result File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/handlers.py", line 118, in post response = await PipelineProcessorManager.instance().process(pipeline) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor.py", line 113, in process res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline) File "/opt/conda/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 213, in process raise RuntimeError('Error pre-processing pipeline {} for engine {} at {}'. RuntimeError: ('Error pre-processing pipeline untitled2 for engine Argo at /tmp/tmpi3sv_ihj/untitled2.tar.gz', "'ComponentParameter' object has no attribute 'type'")

Thankyou.

ptitzler
ptitzler

Regarding the error loading pipeline message: I believe this is a usability issue that we've fixed in version 3.2. The .pipeline file you were trying to open was created using a new release of Elyra. Since your pipeline editor version is older it didn't know how to process the pipeline file. Starting with version 3.2 the pipeline editor will issue a user friendly message that communicates what the problem is and how to solve it (upgrade Elyra).

open pull request

ptitzler wants to merge elyra-ai/examples

ptitzler
ptitzler

Add truncate-file and split-file KFP example custom components

This PR

  • adds two new example KFP custom components
  • Fixes issues in other example KFP custom component READMEs

Example pipeline that utilizes both components:

image

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

How about we defer this to another PR since this is common to all example custom components?

pull request

ptitzler merge to elyra-ai/examples

ptitzler
ptitzler

Add truncate-file and split-file KFP example custom components

This PR

  • adds two new example KFP custom components
  • Fixes issues in other example KFP custom component READMEs

Example pipeline that utilizes both components:

image

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Error running pipeline with custom components

Created a pipeline with "run notebook using papermill" kfp custom component. Notebook parameter of the component is updated with notebook https://github.com/elyra-ai/examples/blob/master/pipelines/kubeflow_pipelines_component_examples/run_notebook_component/a-notebook.ipynb.

Below is error message while running the pipeline.

Error Message : Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 207, in process kfp_argo_compiler.Compiler().compile(pipeline_function, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1179, in compile self._create_and_write_workflow( File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1231, in _create_and_write_workflow workflow = self._create_workflow(pipeline_func, pipeline_name, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1009, in _create_workflow pipeline_func(*args_list, **kwargs_dict) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 194, in pipeline_function = lambda: self._cc_pipeline(pipeline, #8 E731 File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 534, in _cc_pipeline self.log.debug(f"Processing component parameter '{component_property.name}' " AttributeError: 'ComponentParameter' object has no attribute 'type'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute result = await result File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/handlers.py", line 118, in post response = await PipelineProcessorManager.instance().process(pipeline) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor.py", line 113, in process res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline) File "/opt/conda/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 213, in process raise RuntimeError('Error pre-processing pipeline {} for engine {} at {}'. RuntimeError: ('Error pre-processing pipeline untitled2 for engine Argo at /tmp/tmpi3sv_ihj/untitled2.tar.gz', "'ComponentParameter' object has no attribute 'type'")

Thankyou.

ptitzler
ptitzler

We've just released version 3.2.0, which includes significant enhancements for custom components. Could you please give that version a try?

Oct
12
4 days ago
Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Fix validation checks for malformed inputpath in kfp pipelines

Add additional check for missing value when a component uses an inputpath parameter but the value is missing

What changes were proposed in this pull request?

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

Target for 3.3 or 3.2.1 (if there is a 3.2.1 patch)

open pull request

ptitzler wants to merge elyra-ai/elyra

ptitzler
ptitzler

Fix validation checks for malformed inputpath in kfp pipelines

Add additional check for missing value when a component uses an inputpath parameter but the value is missing

What changes were proposed in this pull request?

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler
                                                         message="Node is missing required input source value. "
pull request

ptitzler merge to elyra-ai/elyra

ptitzler
ptitzler

Fix validation checks for malformed inputpath in kfp pipelines

Add additional check for missing value when a component uses an inputpath parameter but the value is missing

What changes were proposed in this pull request?

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
Activity icon
issue

ptitzler issue elyra-ai/elyra

ptitzler
ptitzler

Pipeline validation fails with "AttributeError: 'str' object has no attribute 'keys'"

Describe the issue

Pipeline validation fails with the following error if a runtime-specific pipeline includes a custom component that requires an input source (but has none defined):

packages/elyra/pipeline/handlers.py", line 110, in post
        response = await PipelineValidationManager.instance().validate(pipeline=pipeline_definition)
      File "/opt/anaconda3/envs/ga_32/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 157, in validate
        response=response)
      File "/opt/anaconda3/envs/ga_32/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 325, in _validate_node_properties
        if len(node.get_component_parameter(node_property).keys()) < 2:
    AttributeError: 'str' object has no attribute 'keys'

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Screenshots or log output

Deployment information Describe what you've deployed and how:

  • Elyra version: 3.2.0
push

ptitzler push ptitzler/elyra

ptitzler
ptitzler

Make small changes to KFP DSL export template (#2222)

Fixes issue where exported generic pipeline file fails during submission due to missing authorization credentials

commit sha: 9c0f1a983c7e9c6ba3d66e7540de1180c7915908

push time in 3 days ago
Activity icon
issue

ptitzler issue comment elyra-ai/component-library

ptitzler
ptitzler

Currently only python 3.6 and 3.8 is supported, but the version of Python is 3.7

I got warning and errors: Currently only python 3.6 and 3.8 is supported, in case you need a different version please open an issue at https://github.com/elyra-ai/component-library/issues

The version of Python is 3.7, please help take a look at, thanks. ~/conda/envs/python/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)

ptitzler
ptitzler

My apologies. The author/maintainer of this component is currently on vacation. I'll reach out to him once he returns to get some clarifications.

Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Support KFP with LDAP Dex auth

What changes were proposed in this pull request?

This PR resolves https://github.com/elyra-ai/elyra/issues/2210

  • support LDAP Dex authentication for KFP
  • if both LDAP and staticPasswords authentication are enabled, we use staticPasswords

How was this pull request tested?

This PR was tested locally in the following configurations:

  • staticPasswords enabled only
  • LDAP enabled only
  • staticPasswords and LDAP enabled

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

Is there a way to configure DEX to redirect to a "preferred" auth URL instead of giving the client a choice (which appears to require response parsing based on above observations)?

pull request

ptitzler merge to elyra-ai/elyra

ptitzler
ptitzler

Support KFP with LDAP Dex auth

What changes were proposed in this pull request?

This PR resolves https://github.com/elyra-ai/elyra/issues/2210

  • support LDAP Dex authentication for KFP
  • if both LDAP and staticPasswords authentication are enabled, we use staticPasswords

How was this pull request tested?

This PR was tested locally in the following configurations:

  • staticPasswords enabled only
  • LDAP enabled only
  • staticPasswords and LDAP enabled

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

The most recent changes don't seem to work for the LDAP deployment Alan set up in our test environment. Here's the flow I've observed running some tests on a system that has LDAP and static passwords configured:

  • KFP/DEX responds to GET .../pipeline with

     http://scrip1.fyre.ibm.com:32402/dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=MT...K0G
    
  • a (manually performed) troubleshooting GET request to above URL yields the following HTML (only fragments shown)

        <div class="theme-form-row">
          <a href="/dex/auth/local?req=kvkqnxrira4dxscaxwx66lntl" target="_self">
            <button class="dex-btn theme-btn-provider">
              <span class="dex-btn-icon dex-btn-icon--local"></span>
              <span class="dex-btn-text">Log in with Email</span>
            </button>
          </a>
        </div>
    
        <div class="theme-form-row">
          <a href="/dex/auth/ldap?req=kvkqnxrira4dxscaxwx66lntl" target="_self">
            <button class="dex-btn theme-btn-provider">
              <span class="dex-btn-icon dex-btn-icon--ldap"></span>
              <span class="dex-btn-text">Log in with OpenLDAP</span>
            </button>
          </a>
        </div>    
    

    Note the embedded URLs to the auth endpoints for the two providers: /dex/auth/local?req=kvkqnxrira4dxscaxwx66lntl and /dex/auth/ldap?req=kvkqnxrira4dxscaxwx66lntl

The current rewrite logic in this PR changes the original URL to

http://scrip1.fyre.ibm.com:32402/dex/auth/local?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc& ...

which causes the server to fail.

It appears that additional work is needed.

pull request

ptitzler merge to elyra-ai/elyra

ptitzler
ptitzler

Support KFP with LDAP Dex auth

What changes were proposed in this pull request?

This PR resolves https://github.com/elyra-ai/elyra/issues/2210

  • support LDAP Dex authentication for KFP
  • if both LDAP and staticPasswords authentication are enabled, we use staticPasswords

How was this pull request tested?

This PR was tested locally in the following configurations:

  • staticPasswords enabled only
  • LDAP enabled only
  • staticPasswords and LDAP enabled

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler

The most recent changes don't seem to work for the LDAP deployment Alan set up in out test environment. Here's the flow I've observed running some tests on a system that has LDAP and static passwords configured:

  • KFP/DEX responds to GET .../pipeline with

     http://scrip1.fyre.ibm.com:32402/dex/auth?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc&response_type=code&scope=profile+email+groups+openid&state=MT...K0G
    
  • a (manually performed) troubleshooting GET request to above URL yields the following HTML (only fragments shown)

        <div class="theme-form-row">
          <a href="/dex/auth/local?req=kvkqnxrira4dxscaxwx66lntl" target="_self">
            <button class="dex-btn theme-btn-provider">
              <span class="dex-btn-icon dex-btn-icon--local"></span>
              <span class="dex-btn-text">Log in with Email</span>
            </button>
          </a>
        </div>
    
        <div class="theme-form-row">
          <a href="/dex/auth/ldap?req=kvkqnxrira4dxscaxwx66lntl" target="_self">
            <button class="dex-btn theme-btn-provider">
              <span class="dex-btn-icon dex-btn-icon--ldap"></span>
              <span class="dex-btn-text">Log in with OpenLDAP</span>
            </button>
          </a>
        </div>    
    

    Note the embedded URLs to the auth endpoints for the two providers: /dex/auth/local?req=kvkqnxrira4dxscaxwx66lntl and /dex/auth/ldap?req=kvkqnxrira4dxscaxwx66lntl

The current rewrite logic in this PR changes the original URL to

http://scrip1.fyre.ibm.com:32402/dex/auth/local?client_id=kubeflow-oidc-authservice&redirect_uri=%2Flogin%2Foidc& ...

which causes the server to fail.

It appears that additional work is needed.

Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

Error running pipeline with custom components

Created a pipeline with "run notebook using papermill" kfp custom component. Notebook parameter of the component is updated with notebook https://github.com/elyra-ai/examples/blob/master/pipelines/kubeflow_pipelines_component_examples/run_notebook_component/a-notebook.ipynb.

Below is error message while running the pipeline.

Error Message : Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 207, in process kfp_argo_compiler.Compiler().compile(pipeline_function, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1179, in compile self._create_and_write_workflow( File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1231, in _create_and_write_workflow workflow = self._create_workflow(pipeline_func, pipeline_name, File "/opt/conda/lib/python3.8/site-packages/kfp/compiler/compiler.py", line 1009, in _create_workflow pipeline_func(*args_list, **kwargs_dict) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 194, in pipeline_function = lambda: self._cc_pipeline(pipeline, #8 E731 File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 534, in _cc_pipeline self.log.debug(f"Processing component parameter '{component_property.name}' " AttributeError: 'ComponentParameter' object has no attribute 'type'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/conda/lib/python3.8/site-packages/tornado/web.py", line 1704, in _execute result = await result File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/handlers.py", line 118, in post response = await PipelineProcessorManager.instance().process(pipeline) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor.py", line 113, in process res = await asyncio.get_event_loop().run_in_executor(None, processor.process, pipeline) File "/opt/conda/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "/opt/conda/lib/python3.8/site-packages/elyra/pipeline/processor_kfp.py", line 213, in process raise RuntimeError('Error pre-processing pipeline {} for engine {} at {}'. RuntimeError: ('Error pre-processing pipeline untitled2 for engine Argo at /tmp/tmpi3sv_ihj/untitled2.tar.gz', "'ComponentParameter' object has no attribute 'type'")

Thankyou.

ptitzler
ptitzler

@VMeghanaReddy did you run the example pipeline or did you create your own?

push

ptitzler push ptitzler/elyra

ptitzler
ptitzler

Update 'elyra-pipeline describe' tests

commit sha: 08616e34db447bc96478a269b71c5d51e46a9965

push time in 4 days ago
Oct
11
5 days ago
Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

'elyra-pipeline validate' command fails for generic pipelines

Describe the issue

$ elyra-pipeline validate generic.pipeline

────────────────────────────────────────────────────────────────
 Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ga_31/bin/elyra-pipeline", line 8, in <module>
    sys.exit(pipeline())
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 257, in validate
    _validate_pipeline_definition(pipeline_definition)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 178, in _validate_pipeline_definition
    PipelineValidationManager.instance().validate(pipeline=pipeline_definition))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 152, in validate
    response=response)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 255, in _validate_node_properties
    component_list = await PipelineProcessorManager.instance().get_components(pipeline_runtime)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 98, in get_components
    processor = self._get_processor_for_runtime(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 91, in _get_processor_for_runtime
    processor = self._registry.get_processor(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 76, in get_processor
    raise RuntimeError('Could not find pipeline processor for [{}]'.format(processor_type))
RuntimeError: Could not find pipeline processor for [None]

To Reproduce Steps to reproduce the behavior:

  1. Create a notebook
  2. Create a generic pipeline
  3. Add notebook to pipeline
  4. Save pipeline
  5. Run elyra-pipeline validate ...

Deployment information Describe what you've deployed and how:

  • Elyra version: 3.1+
ptitzler
ptitzler

Behavior in 3.2.0rc2:

$ elyra-pipeline validate generic.pipeline   

────────────────────────────────────────────────────────────────
 Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
 [Error] - Unsupported pipeline runtime 
 [Error] - Unsupported pipeline type 

Error: Pipeline validation FAILED. The pipeline was not submitted for execution.
Activity icon
issue

ptitzler issue elyra-ai/elyra

ptitzler
ptitzler

'elyra-pipeline validate' command fails for generic pipelines

Describe the issue

$ elyra-pipeline validate generic.pipeline

────────────────────────────────────────────────────────────────
 Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ga_31/bin/elyra-pipeline", line 8, in <module>
    sys.exit(pipeline())
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 257, in validate
    _validate_pipeline_definition(pipeline_definition)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 178, in _validate_pipeline_definition
    PipelineValidationManager.instance().validate(pipeline=pipeline_definition))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 152, in validate
    response=response)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 255, in _validate_node_properties
    component_list = await PipelineProcessorManager.instance().get_components(pipeline_runtime)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 98, in get_components
    processor = self._get_processor_for_runtime(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 91, in _get_processor_for_runtime
    processor = self._registry.get_processor(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 76, in get_processor
    raise RuntimeError('Could not find pipeline processor for [{}]'.format(processor_type))
RuntimeError: Could not find pipeline processor for [None]

To Reproduce Steps to reproduce the behavior:

  1. Create a notebook
  2. Create a generic pipeline
  3. Add notebook to pipeline
  4. Save pipeline
  5. Run elyra-pipeline validate ...

Deployment information Describe what you've deployed and how:

  • Elyra version: 3.1+
Activity icon
issue

ptitzler issue elyra-ai/elyra

ptitzler
ptitzler

'elyra-pipeline validate' command fails for generic pipelines

Describe the issue

$ elyra-pipeline validate generic.pipeline

────────────────────────────────────────────────────────────────
 Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ga_31/bin/elyra-pipeline", line 8, in <module>
    sys.exit(pipeline())
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 257, in validate
    _validate_pipeline_definition(pipeline_definition)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 178, in _validate_pipeline_definition
    PipelineValidationManager.instance().validate(pipeline=pipeline_definition))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 152, in validate
    response=response)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 255, in _validate_node_properties
    component_list = await PipelineProcessorManager.instance().get_components(pipeline_runtime)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 98, in get_components
    processor = self._get_processor_for_runtime(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 91, in _get_processor_for_runtime
    processor = self._registry.get_processor(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 76, in get_processor
    raise RuntimeError('Could not find pipeline processor for [{}]'.format(processor_type))
RuntimeError: Could not find pipeline processor for [None]

To Reproduce Steps to reproduce the behavior:

  1. Create a notebook
  2. Create a generic pipeline
  3. Add notebook to pipeline
  4. Save pipeline
  5. Run elyra-pipeline validate ...

Deployment information Describe what you've deployed and how:

  • Elyra version: 3.1+
Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

'elyra-pipeline validate' command fails for generic pipelines

Describe the issue

$ elyra-pipeline validate generic.pipeline

────────────────────────────────────────────────────────────────
 Elyra Pipeline Validation
────────────────────────────────────────────────────────────────

Validating pipeline...
Traceback (most recent call last):
  File "/opt/anaconda3/envs/ga_31/bin/elyra-pipeline", line 8, in <module>
    sys.exit(pipeline())
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 257, in validate
    _validate_pipeline_definition(pipeline_definition)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/cli/pipeline_app.py", line 178, in _validate_pipeline_definition
    PipelineValidationManager.instance().validate(pipeline=pipeline_definition))
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
    return future.result()
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 152, in validate
    response=response)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/validation.py", line 255, in _validate_node_properties
    component_list = await PipelineProcessorManager.instance().get_components(pipeline_runtime)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 98, in get_components
    processor = self._get_processor_for_runtime(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 91, in _get_processor_for_runtime
    processor = self._registry.get_processor(processor_type)
  File "/opt/anaconda3/envs/ga_31/lib/python3.7/site-packages/elyra/pipeline/processor.py", line 76, in get_processor
    raise RuntimeError('Could not find pipeline processor for [{}]'.format(processor_type))
RuntimeError: Could not find pipeline processor for [None]

To Reproduce Steps to reproduce the behavior:

  1. Create a notebook
  2. Create a generic pipeline
  3. Add notebook to pipeline
  4. Save pipeline
  5. Run elyra-pipeline validate ...

Deployment information Describe what you've deployed and how:

  • Elyra version: 3.1+
ptitzler
ptitzler

No longer reproducible using 3.2rc2.

Activity icon
issue

ptitzler issue comment elyra-ai/elyra

ptitzler
ptitzler

elyra-pipeline describe command: list component source dependencies

What changes were proposed in this pull request?

This PR extends the elyra-pipeline describe command by listing (custom) component sources that the pipeline requires.

Example outputs:

$ elyra-pipeline describe io-test.pipeline
...

Name: io-test
Description: test pipeline components
Type: kfp
Version: 5
Nodes: 3
File Dependencies:
    None Listed
Component Dependencies:
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/download-file.yaml
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/count-rows.yaml
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/truncate-file.yaml

JSON formatted:

$ elyra-pipeline describe io-test.pipeline --json

────────────────────────────────────────────────────────────────
 Elyra Pipeline details
────────────────────────────────────────────────────────────────

{
    "name": "io-test",
    "description": "test pipeline components",
    "type": "kfp",
    "version": 5,
    "nodes": 3,
    "component_dependencies": [
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/download-file.yaml",
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/count-rows.yaml",
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/truncate-file.yaml"
    ]
}

Generic pipeline (no custom components used):

$ elyra-pipeline describe generic.pipeline 

Name: generic
Description: None
Type: generic
Version: 5
Nodes: 1
File Dependencies:
    None Listed
Component Dependencies:
    None Listed

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.
ptitzler
ptitzler
pull request

ptitzler pull request elyra-ai/elyra

ptitzler
ptitzler

elyra-pipeline describe command: list component source dependencies

What changes were proposed in this pull request?

This PR extends the elyra-pipeline describe command by listing (custom) component sources that the pipeline requires.

Example outputs:

$ elyra-pipeline describe io-test.pipeline
...

Name: io-test
Description: test pipeline components
Type: kfp
Version: 5
Nodes: 3
File Dependencies:
    None Listed
Component Dependencies:
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/download-file.yaml
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/count-rows.yaml
    - /Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/truncate-file.yaml

JSON formatted:

$ elyra-pipeline describe io-test.pipeline --json

────────────────────────────────────────────────────────────────
 Elyra Pipeline details
────────────────────────────────────────────────────────────────

{
    "name": "io-test",
    "description": "test pipeline components",
    "type": "kfp",
    "version": 5,
    "nodes": 3,
    "component_dependencies": [
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/download-file.yaml",
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/count-rows.yaml",
        "/Users/patti/elyra_dev/workspace/examples/pipelines/run-pipelines-on-kubeflow-pipelines/components/truncate-file.yaml"
    ]
}

Generic pipeline (no custom components used):

$ elyra-pipeline describe generic.pipeline 

Name: generic
Description: None
Type: generic
Version: 5
Nodes: 1
File Dependencies:
    None Listed
Component Dependencies:
    None Listed

How was this pull request tested?

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.