casassg

casassg

I do codes 🖥 I like coffee ☕️

Member Since 7 years ago

@twitter, Boulder, CO

5 organizations

Twitter Project EPIC HackCU HackUPC Team HackAssistant

Experience Points
87
follower
Lessons Completed
140
follow
Lessons Completed
186
stars
Best Reply Awards
69
repos

261 contributions in the last year

casassg Most Used Languages
casassg GitHub Stats

6 Pinned

⚡ Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
⚡ ✏️ Hackathon registration server
⚡ 🗓 Events statically-generated landing page
⚡ ⚙️EPIC Infrastructure backend mono repo.
⚡ 🔎User interface for EPIC Infrastructure
⚡ TFX-Addons is a collection of community projects to build new components, examples, libraries, and tools for TFX. The projects are organized under the auspices of the special interest group, SIG TFX-Addons. Join the group at http://goo.gle/tfx-addons-group
Aug
5
8 hours ago
Activity icon
issue

casassg issue tensorflow/tfx

casassg
casassg

[Discussion] Discrepancy between 1.0.0rc2 and 1.0.0

Hi folks, this is not a major issue, but it caused some headaches on our side.

Seems 1.0.0rc2 had some changes when it was promoted to 1.0.0. My assumption was that the latest rc should contain the same code as the end release.

Not major issue, but would be nice if in the future we can enforce it better :D

Diff: https://github.com/tensorflow/tfx/compare/v1.0.0-rc2...v1.0.0 (this added a new transitive dependency that broke our CI)

Activity icon
issue

casassg issue comment tensorflow/tfx

casassg
casassg

[Discussion] Discrepancy between 1.0.0rc2 and 1.0.0

Hi folks, this is not a major issue, but it caused some headaches on our side.

Seems 1.0.0rc2 had some changes when it was promoted to 1.0.0. My assumption was that the latest rc should contain the same code as the end release.

Not major issue, but would be nice if in the future we can enforce it better :D

Diff: https://github.com/tensorflow/tfx/compare/v1.0.0-rc2...v1.0.0 (this added a new transitive dependency that broke our CI)

casassg
casassg

ty for response, closing for now :D

started
started time in 8 hours ago
Aug
4
1 day ago
Activity icon
issue

casassg issue comment tensorflow/tfx-addons

casassg
casassg

Run linter into sklearn_example to make sure it can be run under CI

Expected Behavior

Currently sklearn pylint/isort/yapf is not working for sklearn_example. We should either consolidate the CI for that and the rest of the project or figure out a good strategy to maitain those.

CC @TheMichaelHu

casassg
casassg

Btw, this is the errors getting in sklearn: https://github.com/tensorflow/tfx-addons/runs/3244621362#step:4:73

YAPF/Isort is mostly stylistic stuff to avoid some of the pylint issues. We can skip them for certain examples (or we can just run them to format it)

Activity icon
issue

casassg issue comment tensorflow/tfx-addons

casassg
casassg

Run linter into sklearn_example to make sure it can be run under CI

Expected Behavior

Currently sklearn pylint/isort/yapf is not working for sklearn_example. We should either consolidate the CI for that and the rest of the project or figure out a good strategy to maitain those.

CC @TheMichaelHu

Activity icon
issue

casassg issue comment tensorflow/tfx-addons

casassg
casassg

Moving pre-commit to a separate workflow

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR
open pull request

casassg wants to merge tensorflow/tfx-addons

casassg
casassg

Moving pre-commit to a separate workflow

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR
pull request

casassg merge to tensorflow/tfx-addons

casassg
casassg

Moving pre-commit to a separate workflow

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR
open pull request

casassg wants to merge tensorflow/tfx-addons

casassg
casassg

[xgb] Example local penguin pipeline using XGBoost in Trainer

This PR adds an example local pipeline that trains and serializes an XGBoost model, as the follow up after https://groups.google.com/u/0/a/tensorflow.org/g/sig-tfx-addons/c/Bx6zvqTVOU0 and the approval of the XGBoost Evaluator project.

In the proposal, we asked for

  • tfx_addons/xgboost_evaluator (evaluator code and tests)
  • examples/xgboost_penguins (example pipelines and tests)

but then only tfx_addons/xgboost_evaluator was created, so I'm putting this example pipeline within this folder.

Local test result

$ pytest
====================================== test session starts ======================================
platform darwin -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/vincentn/venvs/penguin/bin/python
cachedir: .pytest_cache
rootdir: /Users/vincentn/code/tfx-addons, configfile: pyproject.toml
collected 10 items

tfx_addons/mlmd_client/client_test.py::test_pipeline_exists PASSED                        [ 10%]
tfx_addons/mlmd_client/client_test.py::test_get_artifacts PASSED                          [ 20%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::testConstruct PASSED [ 30%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::test_session SKIPPED [ 40%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDo SKIPPED (...) [ 50%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithCache SKIPPED [ 60%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithModuleFile SKIPPED [ 70%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::test_session SKIPPED [ 80%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::testPenguinPipelineLocal PASSED [ 90%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::test_session SKIPPED [100%]
casassg
casassg

Moving this to a separate CI job See #60

pull request

casassg merge to tensorflow/tfx-addons

casassg
casassg

[xgb] Example local penguin pipeline using XGBoost in Trainer

This PR adds an example local pipeline that trains and serializes an XGBoost model, as the follow up after https://groups.google.com/u/0/a/tensorflow.org/g/sig-tfx-addons/c/Bx6zvqTVOU0 and the approval of the XGBoost Evaluator project.

In the proposal, we asked for

  • tfx_addons/xgboost_evaluator (evaluator code and tests)
  • examples/xgboost_penguins (example pipelines and tests)

but then only tfx_addons/xgboost_evaluator was created, so I'm putting this example pipeline within this folder.

Local test result

$ pytest
====================================== test session starts ======================================
platform darwin -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/vincentn/venvs/penguin/bin/python
cachedir: .pytest_cache
rootdir: /Users/vincentn/code/tfx-addons, configfile: pyproject.toml
collected 10 items

tfx_addons/mlmd_client/client_test.py::test_pipeline_exists PASSED                        [ 10%]
tfx_addons/mlmd_client/client_test.py::test_get_artifacts PASSED                          [ 20%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::testConstruct PASSED [ 30%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::test_session SKIPPED [ 40%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDo SKIPPED (...) [ 50%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithCache SKIPPED [ 60%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithModuleFile SKIPPED [ 70%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::test_session SKIPPED [ 80%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::testPenguinPipelineLocal PASSED [ 90%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::test_session SKIPPED [100%]
pull request

casassg merge to tensorflow/tfx-addons

casassg
casassg

[xgb] Example local penguin pipeline using XGBoost in Trainer

This PR adds an example local pipeline that trains and serializes an XGBoost model, as the follow up after https://groups.google.com/u/0/a/tensorflow.org/g/sig-tfx-addons/c/Bx6zvqTVOU0 and the approval of the XGBoost Evaluator project.

In the proposal, we asked for

  • tfx_addons/xgboost_evaluator (evaluator code and tests)
  • examples/xgboost_penguins (example pipelines and tests)

but then only tfx_addons/xgboost_evaluator was created, so I'm putting this example pipeline within this folder.

Local test result

$ pytest
====================================== test session starts ======================================
platform darwin -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/vincentn/venvs/penguin/bin/python
cachedir: .pytest_cache
rootdir: /Users/vincentn/code/tfx-addons, configfile: pyproject.toml
collected 10 items

tfx_addons/mlmd_client/client_test.py::test_pipeline_exists PASSED                        [ 10%]
tfx_addons/mlmd_client/client_test.py::test_get_artifacts PASSED                          [ 20%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::testConstruct PASSED [ 30%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::test_session SKIPPED [ 40%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDo SKIPPED (...) [ 50%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithCache SKIPPED [ 60%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithModuleFile SKIPPED [ 70%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::test_session SKIPPED [ 80%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::testPenguinPipelineLocal PASSED [ 90%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::test_session SKIPPED [100%]
Activity icon
issue

casassg issue comment tensorflow/tfx-addons

casassg
casassg

[xgb] Example local penguin pipeline using XGBoost in Trainer

This PR adds an example local pipeline that trains and serializes an XGBoost model, as the follow up after https://groups.google.com/u/0/a/tensorflow.org/g/sig-tfx-addons/c/Bx6zvqTVOU0 and the approval of the XGBoost Evaluator project.

In the proposal, we asked for

  • tfx_addons/xgboost_evaluator (evaluator code and tests)
  • examples/xgboost_penguins (example pipelines and tests)

but then only tfx_addons/xgboost_evaluator was created, so I'm putting this example pipeline within this folder.

Local test result

$ pytest
====================================== test session starts ======================================
platform darwin -- Python 3.8.11, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /Users/vincentn/venvs/penguin/bin/python
cachedir: .pytest_cache
rootdir: /Users/vincentn/code/tfx-addons, configfile: pyproject.toml
collected 10 items

tfx_addons/mlmd_client/client_test.py::test_pipeline_exists PASSED                        [ 10%]
tfx_addons/mlmd_client/client_test.py::test_get_artifacts PASSED                          [ 20%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::testConstruct PASSED [ 30%]
tfx_addons/schema_curation/component/component_test.py::SchemaCurationTest::test_session SKIPPED [ 40%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDo SKIPPED (...) [ 50%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithCache SKIPPED [ 60%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::testDoWithModuleFile SKIPPED [ 70%]
tfx_addons/schema_curation/component/executor_test.py::ExecutorTest::test_session SKIPPED [ 80%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::testPenguinPipelineLocal PASSED [ 90%]
tfx_addons/xgboost_evaluator/examples/penguin_pipeline_local_e2e_test.py::PenguinPipelineLocalEndToEndTest::test_session SKIPPED [100%]
push

casassg push casassg/tfx-addons

casassg
casassg

Update .pre-commit-config.yaml

commit sha: 290762ee5c2fe46055362d81761ece185580be97

push time in 14 hours ago
Activity icon
issue

casassg issue tensorflow/tfx-addons

casassg
casassg

Run linter into sklearn_example to make sure it can be run under CI

Expected Behavior

Currently sklearn pylint/isort/yapf is not working for sklearn_example. We should either consolidate the CI for that and the rest of the project or figure out a good strategy to maitain those.

CC @TheMichaelHu

push

casassg push casassg/tfx-addons

casassg
casassg

Update .pre-commit-config.yaml

commit sha: 0717eb3bdaf9e0de06e8b26f01a851cd6f966f8e

push time in 14 hours ago
push

casassg push casassg/tfx-addons

casassg
casassg

Update .pre-commit-config.yaml

commit sha: cb1582ecb803c5de8c2547a4e3952bfcc22687d3

push time in 14 hours ago
pull request

casassg pull request tensorflow/tfx-addons

casassg
casassg

Moving pre-commit to a separate workflow

Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

  • Tests pass
  • Appropriate changes to README are included in PR
Activity icon
issue

casassg issue comment tensorflow/tfx

casassg
casassg

[Discussion] Discrepancy between 1.0.0rc2 and 1.0.0

Hi folks, this is not a major issue, but it caused some headaches on our side.

Seems 1.0.0rc2 had some changes when it was promoted to 1.0.0. My assumption was that the latest rc should contain the same code as the end release.

Not major issue, but would be nice if in the future we can enforce it better :D

Diff: https://github.com/tensorflow/tfx/compare/v1.0.0-rc2...v1.0.0 (this added a new transitive dependency that broke our CI)

casassg
casassg

I would say it would be nice to have. Ty for this. It's not a major issue. Mostly been a surprise for us

Aug
3
2 days ago
Activity icon
issue

casassg issue comment tensorflow/tfx

casassg
casassg

Paginating when loading cached results

After having several runs for a pipeline (we reached this by stresstesting our setup), you reach a state where KFP shared MLMD instance is returning too large of response via GRPC when loading previous executions for a certain component.

This could be fixed by instead paginating the get_cached_outputs or just allowing to limit the query to latest 1000 records - https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/metadata.py#L822

See stacktrace: https://gist.github.com/casassg/c67371a2cd1d000db7ade50748aed5b6

casassg
casassg

TFX - 0.26.3 (we haven't finished upgrading yet) KFP - 1.6.0 (checking whats the cluster version atm)

We use KubeflowDagRunner here (since Kubeflow v2 is not yet released to OSS afaik)

Aug
2
3 days ago
close pull request

casassg wants to merge tensorflow/tfx-addons

casassg
casassg

XGBoost Evaluator Project Implementation

Hello! This is our cleaned-up implementation of the XGBoost Evaluator project, which was recently approved by the Google Team. Your input and feedback is much appreciated!

casassg
casassg

Are we using scikit-learn?

pull request

casassg merge to tensorflow/tfx-addons

casassg
casassg

XGBoost Evaluator Project Implementation

Hello! This is our cleaned-up implementation of the XGBoost Evaluator project, which was recently approved by the Google Team. Your input and feedback is much appreciated!

casassg
casassg

As mentioned by Robert, let's move this under tfx_addons/xgboost_evaluator

pull request

casassg merge to tensorflow/tfx-addons

casassg
casassg

XGBoost Evaluator Project Implementation

Hello! This is our cleaned-up implementation of the XGBoost Evaluator project, which was recently approved by the Google Team. Your input and feedback is much appreciated!

casassg
casassg

As mentioned by Robert, let's move this under tfx_addons/xgboost_evaluator

Activity icon
issue

casassg issue comment tensorflow/tfx-addons

casassg
casassg

XGBoost Evaluator Project Implementation

Hello! This is our cleaned-up implementation of the XGBoost Evaluator project, which was recently approved by the Google Team. Your input and feedback is much appreciated!

casassg
casassg

CC @rcrowe-google can you reopen the PR please 😄

Activity icon
issue

casassg issue comment kubeflow/pipelines

casassg
casassg

feat(sdk): Surface kubernetes configuration in container builder

Description of your changes:

Python Kubernetes SDK doesn't handle well tokens that expire. GCP default authentication has set its own custom refresh token, but for other kubernetes API authentication mechanisms it's not trivial to implement.

In order to customize authentication, users of the python sdk can customize the client configuration object to set a custom refresh_api_key_hook and provide it to load_kube_config.

With this change, in ContainerBuilder, we can inject a custom client configuration that will be used by the client to check the kaniko execution.

I avoided setting it for load_incluster_config as usually that uses the internal token provider in the cluster and therefore has less need to be customized. To avoid confusion, I can name it kubernetes_outcluster_configuration?

Related issues: https://github.com/kubernetes-client/python/issues/741

casassg
casassg
Activity icon
issue

casassg issue comment tensorflow/tfx

casassg
casassg

Paginating when loading cached results

After having several runs for a pipeline (we reached this by stresstesting our setup), you reach a state where KFP shared MLMD instance is returning too large of response via GRPC when loading previous executions for a certain component.

This could be fixed by instead paginating the get_cached_outputs or just allowing to limit the query to latest 1000 records - https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/metadata.py#L822

See stacktrace: https://gist.github.com/casassg/c67371a2cd1d000db7ade50748aed5b6

casassg
casassg

CC @zhitaoli we should try to figure out a way to limit the artifacts pulled by TFX to reduce risks when a pipeline has had several runs

Activity icon
issue

casassg issue tensorflow/tfx

casassg
casassg

Paginating when loading cached results

After having several runs for a pipeline (we reached this by stresstesting our setup), you reach a state where KFP shared MLMD instance is returning too large of response via GRPC when loading previous executions for a certain component.

This could be fixed by instead paginating the get_cached_outputs or just allowing to limit the query to latest 1000 records - https://github.com/tensorflow/tfx/blob/master/tfx/orchestration/metadata.py#L822

See stacktrace: https://gist.github.com/casassg/c67371a2cd1d000db7ade50748aed5b6

Activity icon
delete

casassg in casassg/pipelines delete branch patch-1

deleted time in 2 days ago