iknox-fa

iknox-fa

This account is only for contributing to DBT or anything else that should be associated with my work. Personal account: https://github.com/iknox

Member Since 1 year ago

Fishtown Analytics,

Experience Points
2
follower
Lessons Completed
0
follow
Lessons Completed
3
stars
Best Reply Awards
1
repos

203 contributions in the last year

Pinned
⚡ 🍻 Homebrew formulae for installing dbt on macOS
Activity
May
20
3 days ago
Activity icon
issue

iknox-fa issue dbt-labs/dbt-core

iknox-fa
iknox-fa

[Bug] Downstream models do not run correctly after NetworkX 2.8.1

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

Any 1.0.x version of dbt installed after NetworkX 2.8.1 was released does not correctly run downstream models when using selection syntax. For some reason this does not seem to effect 1.1

Expected Behavior

All downstream models are run

Steps To Reproduce

Install dbt- @v1.0.x Create a project with a model_foo and downstream model_bar Run dbt with selection dbt run --select +model_foo+ Observe that only model_foo is run

Relevant log output

No response

Environment

- OS: MacOS
- Python: 3.10
- dbt: 1.0.6

What database are you using dbt with?

postgres

Additional Context

No response

Activity icon
issue

iknox-fa issue pyparsing/pyparsing

iknox-fa
iknox-fa

Install from source fails using latest pip (22.1)

Hi there! It appears that pip 22.1 breaks source installs of pyparsing.
I've tested this with python 3.8 - 3.10 and pyparsing 3.0.7 - 3.0.9 and all have returned the same result seen below.

TBH my python packaging-fu isn't strong enough to tell if this is a pip bug or a pyparsing bug, but I thought it worth filing an issue.

Click to view error
pip install -v --no-deps --no-binary :all: --ignore-installed ~/Downloads/pyparsing-pyparsing_3.0.9
Using pip 22.1 from /Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip (python 3.10)
Processing /Users/iknox/Downloads/pyparsing-pyparsing_3.0.9
  Running command pip subprocess to install build dependencies
  Collecting flit_core<4,>=3.2
    Using cached flit_core-3.7.1.tar.gz (40 kB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: flit_core
    Building wheel for flit_core (pyproject.toml): started
    Building wheel for flit_core (pyproject.toml): finished with status 'done'
    Created wheel for flit_core: filename=flit_core-3.7.1-py3-none-any.whl size=60195 sha256=ae546f54faee8b92a60ab2ee79511b6b694e55cba25351a1f30fdb5cbeda2aee
    Stored in directory: /Users/iknox/Library/Caches/pip/wheels/54/87/76/ea703765f904c482a43161ee927e47ca327bef7cb046b443a8
  Successfully built flit_core
  Installing collected packages: flit_core
  Successfully installed flit_core-3.7.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 340, in run
    requirement_set = resolver.resolve(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__
    super().__init__(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
    dist = self._prepare_distribution()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 421, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 490, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 57, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 45, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 114, in _install_build_reqs
    conflicting, missing = self.req.build_env.check_requirements(build_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/build_env.py", line 178, in check_requirements
    if req.marker is not None and not req.marker.evaluate():
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 304, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 234, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 215, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.
Activity icon
issue

iknox-fa issue comment pyparsing/pyparsing

iknox-fa
iknox-fa

Install from source fails using latest pip (22.1)

Hi there! It appears that pip 22.1 breaks source installs of pyparsing.
I've tested this with python 3.8 - 3.10 and pyparsing 3.0.7 - 3.0.9 and all have returned the same result seen below.

TBH my python packaging-fu isn't strong enough to tell if this is a pip bug or a pyparsing bug, but I thought it worth filing an issue.

Click to view error
pip install -v --no-deps --no-binary :all: --ignore-installed ~/Downloads/pyparsing-pyparsing_3.0.9
Using pip 22.1 from /Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip (python 3.10)
Processing /Users/iknox/Downloads/pyparsing-pyparsing_3.0.9
  Running command pip subprocess to install build dependencies
  Collecting flit_core<4,>=3.2
    Using cached flit_core-3.7.1.tar.gz (40 kB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: flit_core
    Building wheel for flit_core (pyproject.toml): started
    Building wheel for flit_core (pyproject.toml): finished with status 'done'
    Created wheel for flit_core: filename=flit_core-3.7.1-py3-none-any.whl size=60195 sha256=ae546f54faee8b92a60ab2ee79511b6b694e55cba25351a1f30fdb5cbeda2aee
    Stored in directory: /Users/iknox/Library/Caches/pip/wheels/54/87/76/ea703765f904c482a43161ee927e47ca327bef7cb046b443a8
  Successfully built flit_core
  Installing collected packages: flit_core
  Successfully installed flit_core-3.7.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 340, in run
    requirement_set = resolver.resolve(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__
    super().__init__(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
    dist = self._prepare_distribution()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 421, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 490, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 57, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 45, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 114, in _install_build_reqs
    conflicting, missing = self.req.build_env.check_requirements(build_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/build_env.py", line 178, in check_requirements
    if req.marker is not None and not req.marker.evaluate():
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 304, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 234, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 215, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.
iknox-fa
iknox-fa

Thanks for diving in y'all. We're pinned to 22.0.4 for now.

Activity icon
issue

iknox-fa issue dbt-labs/dbt-spark

iknox-fa
iknox-fa

Set default table type to `delta`

Per discussion: https://dbt-labs.slack.com/archives/C017GDLAF7D/p1652996315344739 it may make sense to set the default table type to delta.

May
19
4 days ago
Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-core

iknox-fa
iknox-fa

[CT-668] [Bug] dbt-core capturing KeyboardInterrupt Event with significant delay after pressing ctrl-c

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When pressing ctrl-c while dbt model executing, dbt-core waiting to capture KeyboardInterrupt Event until current model execution completion and also its waiting even after that with significant delay.

My output: 14:48:40 1 of 8 START view model dbt_test.stg_customers.................................. [RUN] 14:48:55 1 of 8 OK created view model dbt_test.stg_customers............................. [OK in 14.96s] 14:48:55 2 of 8 START view model dbt_test.stg_orders..................................... [RUN] 14:49:15 2 of 8 OK created view model dbt_test.stg_orders................................ [OK in 19.97s] 14:49:15 dbt-core adapter: KeyboardInterrupt Event Captured!!! 14:49:15 3 of 8 START view model dbt_test.stg_payments................................... [RUN] 14:49:15 teradata adapter: Canceling Open connections from teradata adapter 14:49:15 teradata adapter: Connection Name : list_schemas, connection state is: closed 14:49:15 teradata adapter: Connection Name : list_None_dbt_test, connection state is: closed 14:49:15 teradata adapter: Connection Name : model.jaffle_shop.stg_payments, connection state is: closed

Here the run was cancelled with ctrl+c after 1 second from starting but the model was running for the full almost 15 seconds despite this and also next model ran for 20 secs.

Expected Behavior

When pressing ctrl-c while dbt model exeucting, dbt-core should capture KeyboardInterrupt Event immediately and it should not wait until model execution.

Steps To Reproduce

  1. Create dbt project with dbt-teradata adapter
  2. Create model with a long runtime, materialised as a table (my example runs about 400 seconds)
  3. Run the model with dbt run and model selector
  4. Wait for model to start executing
  5. Cancel the run with ctrl+c in the terminal
  6. Check the console or log file

Relevant log output

$ dbt  run
14:47:54  Running with dbt=1.0.1
14:47:54  Found 8 models, 20 tests, 0 snapshots, 0 analyses, 197 macros, 0 operations, 3 seed files, 0 sources, 0 exposures, 0 metrics
14:47:54
14:48:40  Concurrency: 1 threads (target='dev')
14:48:40
14:48:40  1 of 8 START view model dbt_test.stg_customers.................................. [RUN]
14:48:55  1 of 8 OK created view model dbt_test.stg_customers............................. [OK in 14.96s]
14:48:55  2 of 8 START view model dbt_test.stg_orders..................................... [RUN]
14:49:15  2 of 8 OK created view model dbt_test.stg_orders................................ [OK in 19.97s]
14:49:15  dbt-core adapter: Satish: KeyboardInterrupt Event Captured!!!
14:49:15  3 of 8 START view model dbt_test.stg_payments................................... [RUN]
14:49:15  teradata adapter:  Canceling Open connections from teradata adapter
14:49:15  teradata adapter:  Connection Name :  list_schemas, connection state is: closed
14:49:15  teradata adapter:  Connection Name :  list_None_dbt_test, connection state is: closed
14:49:15  teradata adapter:  Connection Name :  model.jaffle_shop.stg_payments, connection state is: closed
14:49:26  teradata adapter:  calling teradata adapter cancel method to cancel connection : model.jaffle_shop.stg_payments as state of this connection is open
14:49:26  teradata adapter:  Closing connection name: model.jaffle_shop.stg_payments
14:49:26  teradata adapter: returning all connection names to dbt core ['list_schemas', 'list_None_dbt_test', 'model.jaffle_shop.stg_payments']
14:49:26  CANCEL query PrintCancelLine from dbt-core list_schemas......................... [CANCEL]
14:49:26  CANCEL query PrintCancelLine from dbt-core list_None_dbt_test................... [CANCEL]
14:49:26  CANCEL query PrintCancelLine from dbt-core model.jaffle_shop.stg_payments....... [CANCEL]
14:49:26  Unhandled error while executing model.jaffle_shop.stg_payments
6 is not a valid connection pool handle
14:49:26  3 of 8 ERROR creating view model dbt_test.stg_payments.......................... [ERROR in 11.10s]
14:49:26
14:49:26  Exited because of keyboard interrupt.
14:49:26
14:49:26  Done. PASS=2 WARN=0 ERROR=0 SKIP=0 TOTAL=2
14:49:28  ctrl-c

Environment

- OS: Windows 10 
- Python:Python 3.9.11
- dbt:1.0.1
    Plugins:
     - teradata: 1.0.0a

What database are you using dbt with?

other (mention it in "Additional Context")

Additional Context

database : teradata connector : dbt-teradata (https://github.com/Teradata/dbt-teradata Parent Issue: https://github.com/Teradata/dbt-teradata/issues/35

dbt-teradata adapter supports cancelling connections. this extended dbt sql adatper functionality and it inherits the is_cancelable

iknox-fa
iknox-fa

Hi @SatishChGit Thanks for opening this issue. Unfortunately, replicating this for debugging is a bit of a big ask as we don't have access to Teradata and our development is done almost exclusively on Macs. I will tag this as something to investigate in the future, but prioritizing that work may take quite some time.

Activity icon
issue

iknox-fa issue pyparsing/pyparsing

iknox-fa
iknox-fa

Install from source fails using latest pip (22.1)

Hi there! It appears that pip 22.1 breaks source installs of pyparsing.
I've tested this with python 3.8 - 3.10 and pyparsing 3.0.7 - 3.0.9 and all have returned the same result:

pip install -v --no-deps --no-binary :all: --ignore-installed ~/Downloads/pyparsing-pyparsing_3.0.9
Using pip 22.1 from /Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip (python 3.10)
Processing /Users/iknox/Downloads/pyparsing-pyparsing_3.0.9
  Running command pip subprocess to install build dependencies
  Collecting flit_core<4,>=3.2
    Using cached flit_core-3.7.1.tar.gz (40 kB)
    Getting requirements to build wheel: started
    Getting requirements to build wheel: finished with status 'done'
    Preparing metadata (pyproject.toml): started
    Preparing metadata (pyproject.toml): finished with status 'done'
  Building wheels for collected packages: flit_core
    Building wheel for flit_core (pyproject.toml): started
    Building wheel for flit_core (pyproject.toml): finished with status 'done'
    Created wheel for flit_core: filename=flit_core-3.7.1-py3-none-any.whl size=60195 sha256=ae546f54faee8b92a60ab2ee79511b6b694e55cba25351a1f30fdb5cbeda2aee
    Stored in directory: /Users/iknox/Library/Caches/pip/wheels/54/87/76/ea703765f904c482a43161ee927e47ca327bef7cb046b443a8
  Successfully built flit_core
  Installing collected packages: flit_core
  Successfully installed flit_core-3.7.1
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 340, in run
    requirement_set = resolver.resolve(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 75, in resolve
    collected = self.factory.collect_root_requirements(root_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 506, in collect_root_requirements
    req = self._make_requirement_from_install_req(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 468, in _make_requirement_from_install_req
    cand = self._make_candidate_from_link(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 215, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 291, in __init__
    super().__init__(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 161, in __init__
    self.dist = self._prepare()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 230, in _prepare
    dist = self._prepare_distribution()
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 302, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 421, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 490, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 57, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 45, in prepare_distribution_metadata
    self._install_build_reqs(finder)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/distributions/sdist.py", line 114, in _install_build_reqs
    conflicting, missing = self.req.build_env.check_requirements(build_reqs)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_internal/build_env.py", line 178, in check_requirements
    if req.marker is not None and not req.marker.evaluate():
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 304, in evaluate
    return _evaluate_markers(self._markers, current_environment)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 234, in _evaluate_markers
    lhs_value = _get_env(environment, lhs.value)
  File "/Users/iknox/Projects/python-models/.direnv/python-3.10.0/lib/python3.10/site-packages/pip/_vendor/packaging/markers.py", line 215, in _get_env
    raise UndefinedEnvironmentName(
pip._vendor.packaging.markers.UndefinedEnvironmentName: 'extra' does not exist in evaluation environment.
May
18
5 days ago
Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-core

iknox-fa
iknox-fa

[CT-504] Intermittent test failure in 'selected_resource' test.

Pull request #5001 added a selected resource feature for the context and a test. We are seeing intermittent failures with the error message:


16:08:01  Completed with 2 errors and 0 warnings:
16:08:01  
16:08:01  Compilation Error in model model2 (models/model2.sql)
16:08:01    dbt was unable to infer all dependencies for the model "model2".
16:08:01    This typically happens when ref() is placed within a conditional block.
16:08:01    
16:08:01    To fix this, add the following hint to the top of the model "model2":
16:08:01    
16:08:01    -- depends_on: {{ ref('model1') }}
16:08:01    
16:08:01    > in model model2 (models/model2.sql)
16:08:01    > called by model model2 (models/model2.sql)
16:08:01  
16:08:01  Compilation Error in snapshot cc_all_snapshot (snapshots/my_snapshot.sql)
16:08:01    dbt was unable to infer all dependencies for the model "cc_all_snapshot".
16:08:01    This typically happens when ref() is placed within a conditional block.
16:08:01    
16:08:01    To fix this, add the following hint to the top of the model "cc_all_snapshot":
16:08:01    
16:08:01    -- depends_on: {{ ref('model2') }}
16:08:01    
16:08:01    > in snapshot cc_all_snapshot (snapshots/my_snapshot.sql)
16:08:01    > called by snapshot cc_all_snapshot (snapshots/my_snapshot.sql)

Also having a whole separate file for it in core/dbt/selected_resource.py is kind of overkill. It should be in some other file. In addition we should try hard to implement it in a way that doesn't use a global file variable, because they are a bad idea in general (despite the fact that we have a number of them) and they all need to be reset for tests to run properly (and quite possibly any persistent server code).

iknox-fa
iknox-fa

@gshank This came up in BLG today. We're going to point this with an AC of "make the intermittent errors stop" but not to avoid globals or change the code to live in another place. If you still feel strongly about those issues could you please open new tickets for them?

Activity icon
issue

iknox-fa issue dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[CT-497] rpc silently rounds numeric results >16 digits

Describe the bug

The rpc server stores numeric results in exponential notation to the precision of 16 decimals. That has the effect of silently rounding large numbers (>16 digits).

Steps To Reproduce

Type the following into the dbt Cloud IDE:

select 11111111111111111 as id

Screen Shot 2020-11-18 at 7 15 44 AM

Or send the following RPC request:

{
    "jsonrpc": "2.0",
    "method": "run_sql",
    "id": "2db9a2fe-9a39-41ef-828c-25e04dd6b07d",
    "params": {
        "timeout": 60,
        "sql": "c2VsZWN0IDExMTExMTExMTExMTExMTExMSBhcyBpZAo=",
        "name": "return_int17"
    }
}

And see the result:

...
                "raw_sql": "select 11111111111111111 as id\n",
                "compiled_sql": "select 11111111111111111 as id\n",
...
                "table": {
                    "column_names": [
                        "id"
                    ],
                    "rows": [
                        [
                            1.1111111111111112e+16
                        ]
                    ]
                }
...

Expected behavior

I did not expect my integer to be rounded. At the very least, I expected a clearer warning that it was happening.

System information

I believe this is database-agnostic.

The output of dbt --version:

$ dbt --version
installed version: 0.18.1
   latest version: 0.18.1
Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[CT-497] rpc silently rounds numeric results >16 digits

Describe the bug

The rpc server stores numeric results in exponential notation to the precision of 16 decimals. That has the effect of silently rounding large numbers (>16 digits).

Steps To Reproduce

Type the following into the dbt Cloud IDE:

select 11111111111111111 as id

Screen Shot 2020-11-18 at 7 15 44 AM

Or send the following RPC request:

{
    "jsonrpc": "2.0",
    "method": "run_sql",
    "id": "2db9a2fe-9a39-41ef-828c-25e04dd6b07d",
    "params": {
        "timeout": 60,
        "sql": "c2VsZWN0IDExMTExMTExMTExMTExMTExMSBhcyBpZAo=",
        "name": "return_int17"
    }
}

And see the result:

...
                "raw_sql": "select 11111111111111111 as id\n",
                "compiled_sql": "select 11111111111111111 as id\n",
...
                "table": {
                    "column_names": [
                        "id"
                    ],
                    "rows": [
                        [
                            1.1111111111111112e+16
                        ]
                    ]
                }
...

Expected behavior

I did not expect my integer to be rounded. At the very least, I expected a clearer warning that it was happening.

System information

I believe this is database-agnostic.

The output of dbt --version:

$ dbt --version
installed version: 0.18.1
   latest version: 0.18.1
iknox-fa
iknox-fa

Per team discussions thee isn't a "good" way to do this that doesn't involve an extremely high engineering lift. Closing ticket.

Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-core

iknox-fa
iknox-fa

[CT-416] [Bug] Under certian circumstances releases can be tagged incorrectly without raising errors/warnings

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

During the ill-fated release of 1.1.0b1 we ran into an issue where the github action used in the release process, softprops/[email protected], did not return any indicator that the tag it was trying to push already existed due to an earlier failed release. This left us with a mismatched release/tag that pointed to different commit shas and it also broke docker image builds because they rely on accurate tags.

My initial thought is that we need to ensure the tag doesn't exist before running action-gh-release, however there may be better ways to address the problem.

Expected Behavior

No response

Steps To Reproduce

No response

Relevant log output

No response

Environment

- OS:
- Python:
- dbt:

What database are you using dbt with?

No response

Additional Context

No response

iknox-fa
iknox-fa

@emmyoop This came up in our core BLG meeting. Given that you're doing release work, would this be included in what you're planning? If not let us know so we can point it for future sprints.

May
9
2 weeks ago
Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[CT-161] RPC server goes into infinitely compiling state when cancelling `dbt deps`

Background

In the Cloud IDE, when you cancel a dbt deps command, the rpc server enters a compiling state and never comes out in my testing (after 5+ minutes). This appears to be a bug with dbt core or the RPC server, rather than cloud getting out of sync with the rpc server. The cloud IDe sends a “kill” command when the cancel button is clicked. As the IDE polls on status, it gets “compiling” back every time. So far, I have only been able to get this to happen with deps and no other commands.

Reproduction steps

In dbt Cloud:

  1. Start an IDE session
  2. Make sure you are in a Ready state
  3. Use the command bar to run dbt deps
  4. Hit the Cancel button while it is running
  5. Notice
  6. That the deps command successfully cancels
  7. That the server is now in a compiling state and stays in that state

You can also watch me repro this in this Loom.

Expected behavior

I expect the server to go to compiling momentarily after cancelling, but to then finish compiling and enter either an error or ready state.

iknox-fa
iknox-fa

We have a few issues regarding dbt not handling inturrupts well.. Spike has been made to investigate further: https://github.com/dbt-labs/dbt-rpc/issues/83

Activity icon
issue

iknox-fa issue dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[SPIKE] dbt hangs under certain interrupt conditions

Spike is to investigate a class of issues that occurs when operation of dbt gets interrupted.

See: https://github.com/dbt-labs/dbt-rpc/issues/71

Activity icon
issue

iknox-fa issue comment dbt-labs/homebrew-dbt

iknox-fa
iknox-fa

[CT-496] Error generating package metadata when trying to install dbt

I am receiving the below error when running brew install dbt in terminal. Please advise -- thank you!

Last 15 lines from /Users/katelyn.crawford/Library/Logs/Homebrew/dbt/24.pip:
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/local/Cellar/dbt/0.21.1_1/libexec/bin/python3 /usr/local/Cellar/dbt/0.21.1_1/libexec/lib/python3.8/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/tmp220wuq4u
  cwd: /private/tmp/dbt--dbt-extractor-20220328-18013-wrxfyl/dbt_extractor-0.4.0
  Preparing metadata (pyproject.toml): finished with status 'error'
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
iknox-fa
iknox-fa

This is a user setup issue, not pointed in BLG 5/9/22. Added triage label.

Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[CT-497] rpc silently rounds numeric results >16 digits

Describe the bug

The rpc server stores numeric results in exponential notation to the precision of 16 decimals. That has the effect of silently rounding large numbers (>16 digits).

Steps To Reproduce

Type the following into the dbt Cloud IDE:

select 11111111111111111 as id

Screen Shot 2020-11-18 at 7 15 44 AM

Or send the following RPC request:

{
    "jsonrpc": "2.0",
    "method": "run_sql",
    "id": "2db9a2fe-9a39-41ef-828c-25e04dd6b07d",
    "params": {
        "timeout": 60,
        "sql": "c2VsZWN0IDExMTExMTExMTExMTExMTExMSBhcyBpZAo=",
        "name": "return_int17"
    }
}

And see the result:

...
                "raw_sql": "select 11111111111111111 as id\n",
                "compiled_sql": "select 11111111111111111 as id\n",
...
                "table": {
                    "column_names": [
                        "id"
                    ],
                    "rows": [
                        [
                            1.1111111111111112e+16
                        ]
                    ]
                }
...

Expected behavior

I did not expect my integer to be rounded. At the very least, I expected a clearer warning that it was happening.

System information

I believe this is database-agnostic.

The output of dbt --version:

$ dbt --version
installed version: 0.18.1
   latest version: 0.18.1
iknox-fa
iknox-fa

@jtcohen6 Where else did this come up so we can reference it?

May
6
2 weeks ago
push

iknox-fa push dbt-labs/dbt-core

iknox-fa
iknox-fa

Revert inaccurate mypy change (#5216)

commit sha: ab0c3510ebe70a67fb669b88e6ea36aff2f46339

push time in 2 weeks ago
Activity icon
delete

iknox-fa in dbt-labs/dbt-core delete branch iknox-no-tix-init-test-skip

deleted time in 2 weeks ago
pull request

iknox-fa pull request dbt-labs/dbt-core

iknox-fa
iknox-fa

Revert inaccurate mypy change

No ticket

Description

Some changes made in PR 5171 to version and plugin detection were made in error and this returns them to previous state. The changes were made to correct a failing test but the issue was actually a fundamental problem with the test which was obscured by inconsistencies between editable installs and direct installs.

Checklist

May
5
2 weeks ago
push

iknox-fa push dbt-labs/dbt-core

iknox-fa
iknox-fa

Even more scrubbing (#5152)

  • Even more scrubbing

  • Changelog entry

  • Even more

  • remove reduendent scrub

  • remove reduendent scrub

  • fix encoding issue

  • keep scrubbed log in args

Co-authored-by: Chenyu Li [email protected]

iknox-fa
iknox-fa

Use yaml renderer (with target context) for rendering selectors (#5136)

  • Use yaml renderer (with target context) for rendering selectors

  • Changie

  • Convert cli_vars tests

  • Add test for var in profiles

  • Add test for cli vars in packages

  • Add test for vars in selectors

iknox-fa
iknox-fa
iknox-fa
iknox-fa

Tweak triage-label.yml to trigger off issue labels instead of PR labels (#5168)

  • fix label check

  • fix label filter to go against issues not PRs

iknox-fa
iknox-fa

Ct 488/migrate invalid model tests (#5143)

  • First test completed.
  • Convert and update more test cases.
  • Complete test migration and remove old files.

Co-authored-by: Mila Page [email protected]

iknox-fa
iknox-fa

Add itertools to modules (#5140)

  • GH hygiene: contributing guide, templates, stalebot (#4967)

  • Update contributing guide

  • Update issue + PR templates

  • Stalebot for all issues, no exceptions

  • Update links

  • Missed one

  • PR feedback

  • Update CHANGELOG

Co-authored-by: Jeremy Cohen [email protected]

iknox-fa
iknox-fa

migrate 009_data_tests to new test framework (#5139)

  • Fold so-called 'data' test into new framework with new vocabulary to match.
  • Add missing files including changelog.
  • Remove unneeded Changelog per team policy on test conversions.
  • Refactor test code to better use our pytest framework. Strengthen assertions.

Co-authored-by: Mila Page [email protected]

iknox-fa
iknox-fa

remove extra class and add connection test (#5163)

  • remove extra class and add connection test

  • add project artifact to avoid breaking other tests

  • add comment

iknox-fa
iknox-fa

Add selector method capabilities to selectors (#4827)

iknox-fa
iknox-fa

Fix adding new cols to check_cols in snapshots (#4893)

iknox-fa
iknox-fa

Ct-65 metrics names with spaces (#5173)

  • Convert existing metrics test

  • add non-failing test for names with spaces

  • Raise ParsingException if metrics name contains spaces

  • Remove old metrics tests

iknox-fa
iknox-fa

When parsing 'all_sources' should be a list of unique dirs (#5176)

  • When parsing 'all_sources' should be a list of unique dirs

  • Changie

  • Fix some unit tests of all_source_paths

  • Convert 039_config_tests

  • Remove old 039_config_tests

  • Add test for duplicate directories in 'all_source_files'

iknox-fa
iknox-fa

Update README for dbt-tests-adapter (#5182)

  • Update README for dbt-tests-adapter

  • Add logo

iknox-fa
iknox-fa

Fix: add warning on duplicated yaml keys (#5146)

  • add warning on duplicated yaml keys

  • update structure and tests

  • fix old test schema file

  • add changelog

iknox-fa
iknox-fa

Convert list tests to pytest (#5178)

iknox-fa
iknox-fa

Bump ubuntu from 20.04 to 22.04 (#5141)

  • Bump ubuntu from 20.04 to 22.04
iknox-fa
iknox-fa

remove reference to unused ok to test label (#5149)

iknox-fa
iknox-fa

Adding Skip Changelog label to Version Bump action (#5203)

iknox-fa
iknox-fa

Convert 013_context_vars_tests to context_methods. Move existing cli_vars into context_methods (#5199)

commit sha: 24a867b637ebe92ecf257b1d1037f64bd237bc45

push time in 2 weeks ago
pull request

iknox-fa pull request dbt-labs/dbt-core

iknox-fa
iknox-fa

Revert inaccurate mypy change

No ticket

Description

Some changes made in PR 5171 to version and plugin detection were made in error and this returns them to previous state. The changes were made to correct a failing test but the issue was actually a fundamental problem with the test which was obscured by inconsistencies between editable installs and direct installs.

Checklist

Activity icon
created branch

iknox-fa in dbt-labs/dbt-core create branch iknox-no-tix-init-test-skip

createdAt 2 weeks ago
push

iknox-fa push dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere (#5171)

commit sha: 4480d05cfbe211bb78cdac721ff2664d0a8fc6a7

push time in 2 weeks ago
Activity icon
delete

iknox-fa in dbt-labs/dbt-core delete branch iknox-mypy-everywhere-latest-CT-121-CT-300

deleted time in 2 weeks ago
pull request

iknox-fa pull request dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere

resolves #4805 #4089

Description

  • Mypy was upgraded to 0.942
  • Code changes were made so mypy can be run on all code in the repo (excepting the postgres plugin) using the --strict option

Checklist

Activity icon
issue

iknox-fa issue dbt-labs/dbt-core

iknox-fa
iknox-fa

[CT-300] Upgrading to mypy latest

More in this PR here: https://github.com/dbt-labs/dbt-core/pull/4486

open pull request

iknox-fa wants to merge dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere

resolves #4805 #4089

Description

  • Mypy was upgraded to 0.942
  • Code changes were made so mypy can be run on all code in the repo (excepting the postgres plugin) using the --strict option

Checklist

pull request

iknox-fa merge to dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere

resolves #4805 #4089

Description

  • Mypy was upgraded to 0.942
  • Code changes were made so mypy can be run on all code in the repo (excepting the postgres plugin) using the --strict option

Checklist

pull request

iknox-fa merge to dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere

resolves #4805 #4089

Description

  • Mypy was upgraded to 0.942
  • Code changes were made so mypy can be run on all code in the repo (excepting the postgres plugin) using the --strict option

Checklist

open pull request

iknox-fa wants to merge dbt-labs/dbt-core

iknox-fa
iknox-fa

Update mypy to latest and turn on everywhere

resolves #4805 #4089

Description

  • Mypy was upgraded to 0.942
  • Code changes were made so mypy can be run on all code in the repo (excepting the postgres plugin) using the --strict option

Checklist

iknox-fa
iknox-fa

Yeah, I added the dev reqs to ensure we had access to the various types-<foo> packages for mypy. Changing editable-requirements to requirements is a bit out of scope here, but I don't mind sneaking it in :)

May
3
2 weeks ago
Activity icon
issue

iknox-fa issue comment dbt-labs/dbt-rpc

iknox-fa
iknox-fa

[CT-250] dbt Cloud unable to distinguish multiple exclusion tags while dbt local(CLI) successfully runs

Have found that there seems to be an issue on how dbt cloud is handling command lines for exclusion. I have tried to run the line below, and as you can see in the screenshot it is giving me an error because it builds a table which it should have not built because of the 3 exclusion tags defined.

dbt run --vars 'bigquery_project_id: {variable}' --exclude tag:ignore, tag:lmod_ltv, tag:ltv dbtcloud_run

My expectation was it should have worked the same way as it had when I ran it on my local where it ran successfully and did not build the models having the 3 defined tags. Please see below screenshot for my local run.

dbtlocal_run

Is this a known issue wherein dbt cloud cannot perform multiple tag exclusion? Or what syntax should I have used to apply this for dbt cloud? Is it not able to handle multiple tag exclusions? Basically we are trying to exclude models with the 3 tags.

FYI currently using v1.0

iknox-fa
iknox-fa

@jtcohen6 Can you move this ticket to dbt-rpc? Given it works correctly in a local env, it seems like the likely culprit unless it's entirely a dbt-cloud bug.

Previous