saraleon1

saraleon1

Member Since 3 years ago

Experience Points
1
follower
Lessons Completed
4
follow
Lessons Completed
2
stars
Best Reply Awards
11
repos

10 contributions in the last year

Pinned
⚡ test repo for world cup
⚡ local ecom model repo
Activity
May
21
2 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-2b6c3e-qqhd

createdAt 2 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-jesse-callaway-cxzm

createdAt 2 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-sarah-waterson-f3z8

createdAt 2 days ago
May
20
3 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-385750-jkyp

createdAt 3 days ago
Activity icon
created branch

saraleon1 in looker/ga360_demo create branch dev-emma-chesley-t56g

createdAt 3 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-sanjota-patil-8jzc

createdAt 3 days ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-raghavendra-polanki-h7jv

createdAt 3 days ago
May
17
6 days ago
Activity icon
issue

saraleon1 issue comment dbt-labs/dbt-docs

saraleon1
saraleon1

Slow response times interacting with data lineage chart on large projects

Describe the bug

Users open the data lineage graph by clicking the button at the bottom-right corner of the page. It takes 2-3 seconds for the graph to load. This issue persists beyond the initial load too. Most interactions take 2-3 seconds to complete when several nodes are selected in the lineage graph. Response times are also slow when using selectors; the page becomes briefly unresponsive and keystrokes aren't immediately input to the text field.

Steps To Reproduce

Serve the data catalog for a large dbt project with relatively large manifest.json and catalog.json files; in my example, 300+ models and 1800+ tests generate a 6.2 MB manifest.json and a 1.2 MB catalog.json.

Click the "data lineage chart" button on the bottom-right corner of the page.

See profiling output below for benchmark.

Expected behavior

The data lineage chart should load under 500ms (or some other arbitrary threshold determined by users' tolerance).

Screenshots and log output

The "View Lineage Chart" button:

image

My profiling output:

image

System information

Which database are you using dbt with?

  • postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

installed version: 0.18.0
   latest version: 0.18.1

Your version of dbt is out of date! You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation

Plugins:
  - bigquery: 0.18.0
  - snowflake: 0.18.0
  - redshift: 0.18.0
  - postgres: 0.18.0

The operating system you're using:

The data catalog is served with the base Docker image library/nginx:1.19.0-alpine.

The documentation is generated with the base Docker image library/python:3.7.7-slim-buster.

The output of python --version: 3.7.7

saraleon1
saraleon1

+1 dbt Cloud user - the slow response time of the data lineage chart for their larger project made this feature relatively unusable. Any time they tried to move or modify the chart, it lags and takes a few seconds to load correctly again

May
13
1 week ago
Activity icon
created branch

saraleon1 in looker/ga360_demo create branch dev-chatura-fernando-9dnw

createdAt 1 week ago
Activity icon
created branch

saraleon1 in looker/ga360_demo create branch dev-panos-plastiras-kxj7

createdAt 1 week ago
May
12
1 week ago
Activity icon
created branch

saraleon1 in saraleon1/test_repo create branch dev-ajinkya-rasam-pk8c

createdAt 1 week ago
Activity icon
created branch

saraleon1 in saraleon1/ecomm_local create branch dev-ajinkya-rasam-pk8c

createdAt 1 week ago
Activity icon
created branch

saraleon1 in saraleon1/test_repo create branch dev-leobardo-rodriguez-cavazos-cgs6

createdAt 1 week ago
Activity icon
created branch

saraleon1 in saraleon1/ecomm_local create branch dev-leobardo-rodriguez-cavazos-cgs6

createdAt 1 week ago
Apr
28
3 weeks ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-b050f5-gwkt

createdAt 3 weeks ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-vishal-patel-mphj

createdAt 3 weeks ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-alan-figueroa-jwjs

createdAt 3 weeks ago
Apr
27
3 weeks ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-bd5626-czkm

createdAt 3 weeks ago
Apr
26
3 weeks ago
Activity icon
issue

saraleon1 issue comment dbt-labs/dbt-core

saraleon1
saraleon1

[CT-540] [Bug] 'target' suddenly not available at compile time in selectors.yml

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

After upgrading to dbt v1.0.5, I get the following error when compiling my model:

Compilation Error
  Could not render {{ target.name == 'dev' | as_bool }}: 'target' is undefined

Expected Behavior

The project should compile without error, as it did in earlier post-v1 releases.

Steps To Reproduce

  1. Add a selector like this to selectors.yml
selectors:
  - name: dev_defer_snapshots
    default: "{{ target.name == 'dev' | as_bool }}"
    definition:
      method: fqn
      value: '*'
      exclude:
        - method: config.materialized
          value: snapshot
  1. Compile the project with dbt v1.0.5
  2. Note the error
  3. Comment out the line containing default: "{{ target.name == 'dev' | as_bool }}"
  4. Compile the project
  5. Note no error

Relevant log output

No response

Environment

- OS: MacOS 12.3.1
- Python: 3.9.10
- dbt: 1.0.5

What database are you using dbt with?

redshift

Additional Context

No response

saraleon1
saraleon1

+1 dbt Cloud enterprise customer. This is resulting in a failure to all jobs due to a failed dbt deps due to a 'target undefined' error

Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-aefca4-h4fz

createdAt 3 weeks ago
Apr
23
1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-44380c-nwn4

createdAt 1 month ago
Apr
22
1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-737ab5-ydqg

createdAt 1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-steve-wergeles-2hmh

createdAt 1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-mohammad-nasir-4kjv

createdAt 1 month ago
Activity icon
created branch

saraleon1 in looker/ga360_demo create branch dev-mariko-n-qm9d

createdAt 1 month ago
Apr
15
1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-50ce77-pbcr

createdAt 1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-drishya-pamplaniyil-trhz

createdAt 1 month ago
Apr
12
1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-6afbee-frb6

createdAt 1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-pushkar-goel-rkck

createdAt 1 month ago
Apr
7
1 month ago
Activity icon
created branch

saraleon1 in saraleon1/thesis create branch dev-marketplace-automation-user-ca1962-ykf2

createdAt 1 month ago
Previous