ckadner

ckadner

Software Developer at IBM

Member Since 6 years ago

@IBM, Santa Cruz, CA

Experience Points
60
follower
Lessons Completed
22
follow
Lessons Completed
43
stars
Best Reply Awards
48
repos

418 contributions in the last year

Pinned
⚡ IBM's SystemML Machine Learning
⚡ Experimental project exploring Tekton and KFP integration
⚡ KFP-Notebook is an operator that enable running notebooks as part of a Kubeflow Pipeline
⚡ Data and AI Assets Catalog and Execution Engine
⚡ MLX Katalog is a project to hold the default content samples to bootstrap Machine Learning eXchange.
⚡ Mirror of Apache Spark
Activity
Oct
13
3 days ago
pull request

ckadner merge to conda-forge/kfp-tekton-feedstock

ckadner
ckadner

kfp-tekton v0.8.1

It is very likely that the current package version for this feedstock is out of date. Notes for merging this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version. Checklist before merging this PR:
  • Dependencies have been updated if changed: see upstream
  • Tests have passed
  • Updated license if changed and license_file is packaged

Note that the bot will stop issuing PRs if more than 3 Version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.

NEW: If you want these PRs to be merged automatically, make an issue with @conda-forge-admin,please add bot automerge in the title and merge the resulting PR. This command will add our new bot automerge feature to your feedstock!

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @space/>conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable. Finally, feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/941184753, please use this URL for debugging

Here is a list of all the pending dependencies (and their versions) for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
python 3.9.5 Anaconda-Server Badge

Dependency Analysis

Please note that this analysis is highly experimental. The aim here is to make maintenance easier by inspecting the package's dependencies. Importantly this analysis does not support optional dependencies, please double check those before making changes. If you do not want hinting of this kind ever please add bot: inspection: false to your conda-forge.yml. If you encounter issues with this feature please ping the bot team conda-forge/bot.

Analysis of the source code shows a discrepancy between the library's imports and the package's stated requirements in the meta.yaml.

Packages found by inspection but not in the meta.yaml:

  • python-kubernetes
  • pyyaml
Oct
12
4 days ago
Activity icon
issue

ckadner issue machine-learning-exchange/mlx

ckadner
ckadner

[API] Add ModelMesh client

Is your feature request related to a problem? Please describe. A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

Activity icon
issue

ckadner issue comment machine-learning-exchange/katalog

ckadner
ckadner

Import components from Tekton Hub

  • Identify suitable tasks from Tekton Hub
  • Create a script to convert the Tekton task YAML into a Kubeflow Pipeline component YAML
  • Add the generated component YAML to the Katalog
  • Add the new components to the two MLX catalog_upload.json files under bootstrapper and quickstart

@yhwang @animeshsingh

Activity icon
issue

ckadner issue machine-learning-exchange/mlx

ckadner
ckadner

Find a way to auto-renew ingress TLS cert from IBM Cloud

/lgtm

What happened, @Tomcli ?

We are using the free ingress TLS cert from IBM Cloud. That cert has to be renew manually for every 3 months.

Originally posted by @Tomcli in https://github.com/machine-learning-exchange/mlx/issues/223#issuecomment-941160031

Activity icon
issue

ckadner issue comment machine-learning-exchange/mlx

ckadner
ckadner

Add upload page scroll and fix security issue

Signed-off-by: Andrew-Butler [email protected]

/cc @romeokienzler This PR is for fixing the upload page view issue you mentioned in #184

As well as a dependency upgrade.

ckadner
ckadner
pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Add readme backup for models and datasets

Signed-off-by: Andrew-Butler [email protected]

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Fix sidebar on display on very small screens

Signed-off-by: Andrew-Butler [email protected]

Added the ability to scroll through the sidebar if the sidebar gets too small.

/cc @romeokienzler please review as it has the most issues on your device.

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Add upload page scroll and fix security issue

Signed-off-by: Andrew-Butler [email protected]

/cc @romeokienzler This PR is for fixing the upload page view issue you mentioned in #184

As well as a dependency upgrade.

Oct
11
5 days ago
open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

when I run this with Make or as a Python script without parameters, it should only check, not fix

open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

if from a merge, could you rebase your branch

open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

create a separate target instead:

.PHONY: verify_npm_packages
verify_npm_packages: ## Verify NPM packages
	@python3 tools/python/verify_npm_packages.py
	@echo "[email protected]: OK"
open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

are these changes desired or are they from a merge?

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

Thanks Phu! I have a few comments, questions below

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

Thanks Phu! I have a few comments, questions below

open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

add a line break at the end of file

open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

could you add parameters to do run fix, update packages?

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Npm update packages

Updates the npm packages through the make check_docs_links command.

Phase 1: Finds outdated npm packages If outdated npm packages are found, asks to audit

Phase 2: If auditing, runs npm audit to attempt to update non-breaking changes Checks if there are outdated packages again

Phase 3: If there are still outdated packages, manual inspection is needed Gives steps on how to manually determine which packages are breaking

ckadner
ckadner

needs a line break at the end of file

Oct
9
1 week ago
Activity icon
issue

ckadner issue comment machine-learning-exchange/mlx

ckadner
ckadner

API should reject asset upload when `id` contain invalid characters

Describe the bug

For example, uploading a dataset YAML with id: codenet_langclass will cause issues when the PVC is created for that dataset later on, since Kubernetes resource IDs cannot contain underscore (_) characters (among others).

DataShim response when running the PVC mounting pipeline for a dataset with id: codenet_langclass:

HTTP response body: {
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "Dataset.com.ie.ibm.hpsys \"codenet_langclass\" is invalid: metadata.name: Invalid value: \"codenet_langclass\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')",
  "reason": "Invalid",
  "details": {
    "name": "codenet_langclass",
    "group": "com.ie.ibm.hpsys",
    "kind": "Dataset",
    "causes": [
      {
        "reason": "FieldValueInvalid",
        "message": "Invalid value: \"codenet_langclass\": a DNS-1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')",
        "field": "metadata.name"
      }
    ]
  },
  "code": 422
}

Additional information:

The actual id of the dataset asset in MLX was codenet-langclass, with correctly replaced underscore _ character. However, the component generating the DataShim metadata used the original id field from the YAML file, with the id containing the underscore character.

FYI, here is a quick test of the sanitize_k8s_name function:

python -c "
from kfp_tekton.compiler._k8s_helper import sanitize_k8s_name; 
print(sanitize_k8s_name('under_score-dash'));
"
under-score-dash

Proposed fix:

Any _upload_... method in api/server/swagger_server/controllers_impl/XXX_service_controller_impl.py which takes an id (or similar field like model_identifier) from the uploaded YAML file to generate the asset id should call a new validate_id() method similar to the swagger_server.controllers_impl.validate_parameters function in api/server/swagger_server/controllers_impl/__init__.py

ckadner
ckadner

@vedantgannu -- I updated the description of this issue with more details and a proposed fix. Let me know if you have any questions. Thanks!

pull request

ckadner merge to kubeflow/kfp-tekton

ckadner
ckadner

Relax kfp sdk dependencies

Which issue is resolved by this Pull Request: Resolves #745

Description of your changes: Relax kfp sdk dependencies because a new bug was introduced in the 1.7.2 SDK release. We are not updating to 1.8.x SDK yet because it is still in active development and also has different bugs in v2 dsl.

We will cut a 1.0.1 sdk release to address these bug fixes similar to how kfp does for their sdk release. Environment tested:

  • Python Version (use python --version):
  • Tekton Version (use tkn version):
  • Kubernetes Version (use kubectl version):
  • OS (e.g. from /etc/os-release):

Checklist:

Oct
8
1 week ago
pull request

ckadner merge to kubeflow/kfp-tekton

ckadner
ckadner

Relax kfp sdk dependencies

Which issue is resolved by this Pull Request: Resolves #745

Description of your changes: Relax kfp sdk dependencies because a new bug was introduced in the 1.7.2 SDK release. We are not updating to 1.8.x SDK yet because it is still in active development and also has different bugs in v2 dsl.

We will cut a 1.0.1 sdk release to address these bug fixes similar to how kfp does for their sdk release. Environment tested:

  • Python Version (use python --version):
  • Tekton Version (use tkn version):
  • Kubernetes Version (use kubectl version):
  • OS (e.g. from /etc/os-release):

Checklist:

ckadner
ckadner

Two more references to version 1.0.0 need to be updated:

Found Occurrences in Directory /Users/ckadner/PycharmProjects/kfp-tekton_ckadner/sdk  (2 usages found)
    kfp-tekton  (2 usages found)
        sdk  (2 usages found)
            python  (2 usages found)
                tests  (1 usage found)
                    README.md  (1 usage found)
                        ln:144 the `kfp_tekton` `1.0.0` compiler code is based on the `kfp` SDK compiler version `1.7.2`.
                README.md  (1 usage found)
                    ln:111 specific version of the `kfp` SDK compiler. As of now the `kfp-tekton` SDK version is `1.0.0` which is aligned with KFP

Eight more references to version 1.7.2 need to be updated:

Occurrences of '1.7.2' in Directory /kfp-tekton/sdk
(8 usages found)
    kfp-tekton  (8 usages found)
        sdk  (8 usages found)
            python  (8 usages found)
                tests  (5 usages found)
                    README.md  (3 usages found)
                        ln:30 KFP SDK version: 1.7.2
                        ln:143 **Note:** The reports above were created for the pipeline scripts found in KFP SDK version `1.7.2` since
                        ln:144 the `kfp_tekton` `1.0.0` compiler code is based on the `kfp` SDK compiler version `1.7.2`.
                    test_kfp_samples.sh  (2 usages found)
                        ln:55 -v|--kfp-version)          KFP_VERSION="$2";            shift 2 ;;  # KFP SDK version, default: 1.7.2
                        ln:67 KFP_SDK_VERSION=${KFP_VERSION:-1.7.2}
                README.md  (2 usages found)
                    ln:112 SDK version [`1.7.2`](https://pypi.org/project/kfp/1.7.2/).
                    ln:112 SDK version [`1.7.2`](https://pypi.org/project/kfp/1.7.2/).
                requirements.txt  (1 usage found)
                    ln:67 kfp==1.7.2
pull request

ckadner merge to kubeflow/kfp-tekton

ckadner
ckadner

Relax kfp sdk dependencies

Which issue is resolved by this Pull Request: Resolves #745

Description of your changes: Relax kfp sdk dependencies because a new bug was introduced in the 1.7.2 SDK release. We are not updating to 1.8.x SDK yet because it is still in active development and also has different bugs in v2 dsl.

We will cut a 1.0.1 sdk release to address these bug fixes similar to how kfp does for their sdk release. Environment tested:

  • Python Version (use python --version):
  • Tekton Version (use tkn version):
  • Kubernetes Version (use kubectl version):
  • OS (e.g. from /etc/os-release):

Checklist:

ckadner
ckadner

Two more references to version 1.0.0 need to be updated:

Found Occurrences in Directory /Users/ckadner/PycharmProjects/kfp-tekton_ckadner/sdk  (2 usages found)
    kfp-tekton  (2 usages found)
        sdk  (2 usages found)
            python  (2 usages found)
                tests  (1 usage found)
                    README.md  (1 usage found)
                        ln:144 the `kfp_tekton` `1.0.0` compiler code is based on the `kfp` SDK compiler version `1.7.2`.
                README.md  (1 usage found)
                    ln:111 specific version of the `kfp` SDK compiler. As of now the `kfp-tekton` SDK version is `1.0.0` which is aligned with KFP

Eight more references to version 1.7.2 need to be updated:

Occurrences of '1.7.2' in Directory /kfp-tekton/sdk
(8 usages found)
    kfp-tekton  (8 usages found)
        sdk  (8 usages found)
            python  (8 usages found)
                tests  (5 usages found)
                    README.md  (3 usages found)
                        30 KFP SDK version: 1.7.2
                        143 **Note:** The reports above were created for the pipeline scripts found in KFP SDK version `1.7.2` since
                        144 the `kfp_tekton` `1.0.0` compiler code is based on the `kfp` SDK compiler version `1.7.2`.
                    test_kfp_samples.sh  (2 usages found)
                        55 -v|--kfp-version)          KFP_VERSION="$2";            shift 2 ;;  # KFP SDK version, default: 1.7.2
                        67 KFP_SDK_VERSION=${KFP_VERSION:-1.7.2}
                README.md  (2 usages found)
                    112 SDK version [`1.7.2`](https://pypi.org/project/kfp/1.7.2/).
                    112 SDK version [`1.7.2`](https://pypi.org/project/kfp/1.7.2/).
                requirements.txt  (1 usage found)
                    67 kfp==1.7.2
Oct
5
1 week ago
Activity icon
issue

ckadner issue comment machine-learning-exchange/katalog

ckadner
ckadner

Future MAX and DAX assets

List of MAX models to be migrated into MLX:

  • CodeNet code complexity analysis (currently prototype)
  • ...

List of DAX datasets to be integrated into MLX:

  • FinTabNet - 15 GB
  • Genomics (2 datasets) - 80 GB -> need to confirm if we need this
  • ...

@kmh4321

ckadner
ckadner

@kmh4321 -- could you provide a tentative list of models and datasets you are planning to integrate into the MLX catalog?

Activity icon
issue

ckadner issue machine-learning-exchange/katalog

ckadner
ckadner

Future MAX and DAX assets

List of MAX models to be migrated into MLX:

  • ...

List of DAX datasets to be integrated into MLX:

  • ...

@kmh4321

Activity icon
issue

ckadner issue comment machine-learning-exchange/mlx

ckadner
ckadner

Add `id` field to notebook YAML

Currently the notebook YAMLs do not have an id or notebook_identifier field. This makes it hard to reference a notebook as a related_asset from other assets, like datasets.

Related issues:

  • machine-learning-exchange/katalog#35

Additional information:

The code to be changed in the MLX API:

https://github.com/machine-learning-exchange/mlx/blob/3da86c777aba636f930a9b8e19b488d08b0031fa/api/server/swagger_server/controllers_impl/notebook_service_controller_impl.py#L398

... add or yaml_dict.get("id") here:

    notebook_id = existing_id or yaml_dict.get("id") or generate_id(name=name or yaml_dict["name"])
ckadner
ckadner

@JAulet -- I added some more info in the description.

Activity icon
issue

ckadner issue comment machine-learning-exchange/katalog

ckadner
ckadner

Rename MAX/DAX markdown files to README.md

So that they get displayed in Github when users land in the respective folders.

  • Rename the *.md files to README.md under dataset-samples/* and model-samples/*
  • Update the readme_url fields in the accompanying *.yaml files
Oct
4
1 week ago
open pull request

ckadner wants to merge machine-learning-exchange/mlx

ckadner
ckadner

Add readme backup for models and datasets

Signed-off-by: Andrew-Butler [email protected]

ckadner
ckadner

@drewbutlerbb4 -- are you still working on this PR or is it ready to be merged?

pull request

ckadner merge to machine-learning-exchange/mlx

ckadner
ckadner

Add readme backup for models and datasets

Signed-off-by: Andrew-Butler [email protected]

Oct
1
2 weeks ago
started
started time in 2 weeks ago