Describe the bug
Users open the data lineage graph by clicking the button at the bottom-right corner of the page. It takes 2-3 seconds for the graph to load. This issue persists beyond the initial load too. Most interactions take 2-3 seconds to complete when several nodes are selected in the lineage graph. Response times are also slow when using selectors; the page becomes briefly unresponsive and keystrokes aren't immediately input to the text field.
Steps To Reproduce
Serve the data catalog for a large dbt project with relatively large
catalog.json files; in my example, 300+ models and 1800+ tests generate a 6.2 MB
manifest.json and a 1.2 MB
Click the "data lineage chart" button on the bottom-right corner of the page.
See profiling output below for benchmark.
The data lineage chart should load under 500ms (or some other arbitrary threshold determined by users' tolerance).
Screenshots and log output
The "View Lineage Chart" button:
My profiling output:
Which database are you using dbt with?
- other (specify: ____________)
The output of
installed version: 0.18.0 latest version: 0.18.1 Your version of dbt is out of date! You can find instructions for upgrading here: https://docs.getdbt.com/docs/installation Plugins: - bigquery: 0.18.0 - snowflake: 0.18.0 - redshift: 0.18.0 - postgres: 0.18.0
The operating system you're using:
The data catalog is served with the base Docker image library/nginx:1.19.0-alpine.
The documentation is generated with the base Docker image library/python:3.7.7-slim-buster.
The output of
python --version: 3.7.7