This is page 36 of 45. Use http://codebase.md/googleapis/genai-toolbox?lines=true&page={x} to view the full context.
# Directory Structure
```
├── .ci
│ ├── continuous.release.cloudbuild.yaml
│ ├── generate_release_table.sh
│ ├── integration.cloudbuild.yaml
│ ├── quickstart_test
│ │ ├── go.integration.cloudbuild.yaml
│ │ ├── js.integration.cloudbuild.yaml
│ │ ├── py.integration.cloudbuild.yaml
│ │ ├── run_go_tests.sh
│ │ ├── run_js_tests.sh
│ │ ├── run_py_tests.sh
│ │ └── setup_hotels_sample.sql
│ ├── test_with_coverage.sh
│ └── versioned.release.cloudbuild.yaml
├── .github
│ ├── auto-label.yaml
│ ├── blunderbuss.yml
│ ├── CODEOWNERS
│ ├── header-checker-lint.yml
│ ├── ISSUE_TEMPLATE
│ │ ├── bug_report.yml
│ │ ├── config.yml
│ │ ├── feature_request.yml
│ │ └── question.yml
│ ├── label-sync.yml
│ ├── labels.yaml
│ ├── PULL_REQUEST_TEMPLATE.md
│ ├── release-please.yml
│ ├── renovate.json5
│ ├── sync-repo-settings.yaml
│ └── workflows
│ ├── cloud_build_failure_reporter.yml
│ ├── deploy_dev_docs.yaml
│ ├── deploy_previous_version_docs.yaml
│ ├── deploy_versioned_docs.yaml
│ ├── docs_deploy.yaml
│ ├── docs_preview_clean.yaml
│ ├── docs_preview_deploy.yaml
│ ├── lint.yaml
│ ├── schedule_reporter.yml
│ ├── sync-labels.yaml
│ └── tests.yaml
├── .gitignore
├── .gitmodules
├── .golangci.yaml
├── .hugo
│ ├── archetypes
│ │ └── default.md
│ ├── assets
│ │ ├── icons
│ │ │ └── logo.svg
│ │ └── scss
│ │ ├── _styles_project.scss
│ │ └── _variables_project.scss
│ ├── go.mod
│ ├── go.sum
│ ├── hugo.toml
│ ├── layouts
│ │ ├── _default
│ │ │ └── home.releases.releases
│ │ ├── index.llms-full.txt
│ │ ├── index.llms.txt
│ │ ├── partials
│ │ │ ├── hooks
│ │ │ │ └── head-end.html
│ │ │ ├── navbar-version-selector.html
│ │ │ ├── page-meta-links.html
│ │ │ └── td
│ │ │ └── render-heading.html
│ │ ├── robot.txt
│ │ └── shortcodes
│ │ ├── include.html
│ │ ├── ipynb.html
│ │ └── regionInclude.html
│ ├── package-lock.json
│ ├── package.json
│ └── static
│ ├── favicons
│ │ ├── android-chrome-192x192.png
│ │ ├── android-chrome-512x512.png
│ │ ├── apple-touch-icon.png
│ │ ├── favicon-16x16.png
│ │ ├── favicon-32x32.png
│ │ └── favicon.ico
│ └── js
│ └── w3.js
├── CHANGELOG.md
├── cmd
│ ├── options_test.go
│ ├── options.go
│ ├── root_test.go
│ ├── root.go
│ └── version.txt
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── DEVELOPER.md
├── Dockerfile
├── docs
│ └── en
│ ├── _index.md
│ ├── about
│ │ ├── _index.md
│ │ └── faq.md
│ ├── concepts
│ │ ├── _index.md
│ │ └── telemetry
│ │ ├── index.md
│ │ ├── telemetry_flow.png
│ │ └── telemetry_traces.png
│ ├── getting-started
│ │ ├── _index.md
│ │ ├── colab_quickstart.ipynb
│ │ ├── configure.md
│ │ ├── introduction
│ │ │ ├── _index.md
│ │ │ └── architecture.png
│ │ ├── local_quickstart_go.md
│ │ ├── local_quickstart_js.md
│ │ ├── local_quickstart.md
│ │ ├── mcp_quickstart
│ │ │ ├── _index.md
│ │ │ ├── inspector_tools.png
│ │ │ └── inspector.png
│ │ └── quickstart
│ │ ├── go
│ │ │ ├── genAI
│ │ │ │ ├── go.mod
│ │ │ │ ├── go.sum
│ │ │ │ └── quickstart.go
│ │ │ ├── genkit
│ │ │ │ ├── go.mod
│ │ │ │ ├── go.sum
│ │ │ │ └── quickstart.go
│ │ │ ├── langchain
│ │ │ │ ├── go.mod
│ │ │ │ ├── go.sum
│ │ │ │ └── quickstart.go
│ │ │ ├── openAI
│ │ │ │ ├── go.mod
│ │ │ │ ├── go.sum
│ │ │ │ └── quickstart.go
│ │ │ └── quickstart_test.go
│ │ ├── golden.txt
│ │ ├── js
│ │ │ ├── genAI
│ │ │ │ ├── package-lock.json
│ │ │ │ ├── package.json
│ │ │ │ └── quickstart.js
│ │ │ ├── genkit
│ │ │ │ ├── package-lock.json
│ │ │ │ ├── package.json
│ │ │ │ └── quickstart.js
│ │ │ ├── langchain
│ │ │ │ ├── package-lock.json
│ │ │ │ ├── package.json
│ │ │ │ └── quickstart.js
│ │ │ ├── llamaindex
│ │ │ │ ├── package-lock.json
│ │ │ │ ├── package.json
│ │ │ │ └── quickstart.js
│ │ │ └── quickstart.test.js
│ │ ├── python
│ │ │ ├── __init__.py
│ │ │ ├── adk
│ │ │ │ ├── quickstart.py
│ │ │ │ └── requirements.txt
│ │ │ ├── core
│ │ │ │ ├── quickstart.py
│ │ │ │ └── requirements.txt
│ │ │ ├── langchain
│ │ │ │ ├── quickstart.py
│ │ │ │ └── requirements.txt
│ │ │ ├── llamaindex
│ │ │ │ ├── quickstart.py
│ │ │ │ └── requirements.txt
│ │ │ └── quickstart_test.py
│ │ └── shared
│ │ ├── cloud_setup.md
│ │ ├── configure_toolbox.md
│ │ └── database_setup.md
│ ├── how-to
│ │ ├── _index.md
│ │ ├── connect_via_geminicli.md
│ │ ├── connect_via_mcp.md
│ │ ├── connect-ide
│ │ │ ├── _index.md
│ │ │ ├── alloydb_pg_admin_mcp.md
│ │ │ ├── alloydb_pg_mcp.md
│ │ │ ├── bigquery_mcp.md
│ │ │ ├── cloud_sql_mssql_admin_mcp.md
│ │ │ ├── cloud_sql_mssql_mcp.md
│ │ │ ├── cloud_sql_mysql_admin_mcp.md
│ │ │ ├── cloud_sql_mysql_mcp.md
│ │ │ ├── cloud_sql_pg_admin_mcp.md
│ │ │ ├── cloud_sql_pg_mcp.md
│ │ │ ├── firestore_mcp.md
│ │ │ ├── looker_mcp.md
│ │ │ ├── mssql_mcp.md
│ │ │ ├── mysql_mcp.md
│ │ │ ├── neo4j_mcp.md
│ │ │ ├── postgres_mcp.md
│ │ │ ├── spanner_mcp.md
│ │ │ └── sqlite_mcp.md
│ │ ├── deploy_docker.md
│ │ ├── deploy_gke.md
│ │ ├── deploy_toolbox.md
│ │ ├── export_telemetry.md
│ │ └── toolbox-ui
│ │ ├── edit-headers.gif
│ │ ├── edit-headers.png
│ │ ├── index.md
│ │ ├── optional-param-checked.png
│ │ ├── optional-param-unchecked.png
│ │ ├── run-tool.gif
│ │ ├── tools.png
│ │ └── toolsets.png
│ ├── reference
│ │ ├── _index.md
│ │ ├── cli.md
│ │ └── prebuilt-tools.md
│ ├── resources
│ │ ├── _index.md
│ │ ├── authServices
│ │ │ ├── _index.md
│ │ │ └── google.md
│ │ ├── sources
│ │ │ ├── _index.md
│ │ │ ├── alloydb-admin.md
│ │ │ ├── alloydb-pg.md
│ │ │ ├── bigquery.md
│ │ │ ├── bigtable.md
│ │ │ ├── cassandra.md
│ │ │ ├── clickhouse.md
│ │ │ ├── cloud-monitoring.md
│ │ │ ├── cloud-sql-admin.md
│ │ │ ├── cloud-sql-mssql.md
│ │ │ ├── cloud-sql-mysql.md
│ │ │ ├── cloud-sql-pg.md
│ │ │ ├── couchbase.md
│ │ │ ├── dataplex.md
│ │ │ ├── dgraph.md
│ │ │ ├── firebird.md
│ │ │ ├── firestore.md
│ │ │ ├── http.md
│ │ │ ├── looker.md
│ │ │ ├── mongodb.md
│ │ │ ├── mssql.md
│ │ │ ├── mysql.md
│ │ │ ├── neo4j.md
│ │ │ ├── oceanbase.md
│ │ │ ├── oracle.md
│ │ │ ├── postgres.md
│ │ │ ├── redis.md
│ │ │ ├── spanner.md
│ │ │ ├── sqlite.md
│ │ │ ├── tidb.md
│ │ │ ├── trino.md
│ │ │ ├── valkey.md
│ │ │ └── yugabytedb.md
│ │ └── tools
│ │ ├── _index.md
│ │ ├── alloydb
│ │ │ ├── _index.md
│ │ │ ├── alloydb-create-cluster.md
│ │ │ ├── alloydb-create-instance.md
│ │ │ ├── alloydb-create-user.md
│ │ │ ├── alloydb-get-cluster.md
│ │ │ ├── alloydb-get-instance.md
│ │ │ ├── alloydb-get-user.md
│ │ │ ├── alloydb-list-clusters.md
│ │ │ ├── alloydb-list-instances.md
│ │ │ ├── alloydb-list-users.md
│ │ │ └── alloydb-wait-for-operation.md
│ │ ├── alloydbainl
│ │ │ ├── _index.md
│ │ │ └── alloydb-ai-nl.md
│ │ ├── bigquery
│ │ │ ├── _index.md
│ │ │ ├── bigquery-analyze-contribution.md
│ │ │ ├── bigquery-conversational-analytics.md
│ │ │ ├── bigquery-execute-sql.md
│ │ │ ├── bigquery-forecast.md
│ │ │ ├── bigquery-get-dataset-info.md
│ │ │ ├── bigquery-get-table-info.md
│ │ │ ├── bigquery-list-dataset-ids.md
│ │ │ ├── bigquery-list-table-ids.md
│ │ │ ├── bigquery-search-catalog.md
│ │ │ └── bigquery-sql.md
│ │ ├── bigtable
│ │ │ ├── _index.md
│ │ │ └── bigtable-sql.md
│ │ ├── cassandra
│ │ │ ├── _index.md
│ │ │ └── cassandra-cql.md
│ │ ├── clickhouse
│ │ │ ├── _index.md
│ │ │ ├── clickhouse-execute-sql.md
│ │ │ ├── clickhouse-list-databases.md
│ │ │ ├── clickhouse-list-tables.md
│ │ │ └── clickhouse-sql.md
│ │ ├── cloudmonitoring
│ │ │ ├── _index.md
│ │ │ └── cloud-monitoring-query-prometheus.md
│ │ ├── cloudsql
│ │ │ ├── _index.md
│ │ │ ├── cloudsqlcreatedatabase.md
│ │ │ ├── cloudsqlcreateusers.md
│ │ │ ├── cloudsqlgetinstances.md
│ │ │ ├── cloudsqllistdatabases.md
│ │ │ ├── cloudsqllistinstances.md
│ │ │ ├── cloudsqlmssqlcreateinstance.md
│ │ │ ├── cloudsqlmysqlcreateinstance.md
│ │ │ ├── cloudsqlpgcreateinstances.md
│ │ │ └── cloudsqlwaitforoperation.md
│ │ ├── couchbase
│ │ │ ├── _index.md
│ │ │ └── couchbase-sql.md
│ │ ├── dataform
│ │ │ ├── _index.md
│ │ │ └── dataform-compile-local.md
│ │ ├── dataplex
│ │ │ ├── _index.md
│ │ │ ├── dataplex-lookup-entry.md
│ │ │ ├── dataplex-search-aspect-types.md
│ │ │ └── dataplex-search-entries.md
│ │ ├── dgraph
│ │ │ ├── _index.md
│ │ │ └── dgraph-dql.md
│ │ ├── firebird
│ │ │ ├── _index.md
│ │ │ ├── firebird-execute-sql.md
│ │ │ └── firebird-sql.md
│ │ ├── firestore
│ │ │ ├── _index.md
│ │ │ ├── firestore-add-documents.md
│ │ │ ├── firestore-delete-documents.md
│ │ │ ├── firestore-get-documents.md
│ │ │ ├── firestore-get-rules.md
│ │ │ ├── firestore-list-collections.md
│ │ │ ├── firestore-query-collection.md
│ │ │ ├── firestore-query.md
│ │ │ ├── firestore-update-document.md
│ │ │ └── firestore-validate-rules.md
│ │ ├── http
│ │ │ ├── _index.md
│ │ │ └── http.md
│ │ ├── looker
│ │ │ ├── _index.md
│ │ │ ├── looker-add-dashboard-element.md
│ │ │ ├── looker-conversational-analytics.md
│ │ │ ├── looker-get-dashboards.md
│ │ │ ├── looker-get-dimensions.md
│ │ │ ├── looker-get-explores.md
│ │ │ ├── looker-get-filters.md
│ │ │ ├── looker-get-looks.md
│ │ │ ├── looker-get-measures.md
│ │ │ ├── looker-get-models.md
│ │ │ ├── looker-get-parameters.md
│ │ │ ├── looker-health-analyze.md
│ │ │ ├── looker-health-pulse.md
│ │ │ ├── looker-health-vacuum.md
│ │ │ ├── looker-make-dashboard.md
│ │ │ ├── looker-make-look.md
│ │ │ ├── looker-query-sql.md
│ │ │ ├── looker-query-url.md
│ │ │ ├── looker-query.md
│ │ │ └── looker-run-look.md
│ │ ├── mongodb
│ │ │ ├── _index.md
│ │ │ ├── mongodb-aggregate.md
│ │ │ ├── mongodb-delete-many.md
│ │ │ ├── mongodb-delete-one.md
│ │ │ ├── mongodb-find-one.md
│ │ │ ├── mongodb-find.md
│ │ │ ├── mongodb-insert-many.md
│ │ │ ├── mongodb-insert-one.md
│ │ │ ├── mongodb-update-many.md
│ │ │ └── mongodb-update-one.md
│ │ ├── mssql
│ │ │ ├── _index.md
│ │ │ ├── mssql-execute-sql.md
│ │ │ ├── mssql-list-tables.md
│ │ │ └── mssql-sql.md
│ │ ├── mysql
│ │ │ ├── _index.md
│ │ │ ├── mysql-execute-sql.md
│ │ │ ├── mysql-list-active-queries.md
│ │ │ ├── mysql-list-table-fragmentation.md
│ │ │ ├── mysql-list-tables-missing-unique-indexes.md
│ │ │ ├── mysql-list-tables.md
│ │ │ └── mysql-sql.md
│ │ ├── neo4j
│ │ │ ├── _index.md
│ │ │ ├── neo4j-cypher.md
│ │ │ ├── neo4j-execute-cypher.md
│ │ │ └── neo4j-schema.md
│ │ ├── oceanbase
│ │ │ ├── _index.md
│ │ │ ├── oceanbase-execute-sql.md
│ │ │ └── oceanbase-sql.md
│ │ ├── oracle
│ │ │ ├── _index.md
│ │ │ ├── oracle-execute-sql.md
│ │ │ └── oracle-sql.md
│ │ ├── postgres
│ │ │ ├── _index.md
│ │ │ ├── postgres-execute-sql.md
│ │ │ ├── postgres-list-active-queries.md
│ │ │ ├── postgres-list-available-extensions.md
│ │ │ ├── postgres-list-installed-extensions.md
│ │ │ ├── postgres-list-tables.md
│ │ │ └── postgres-sql.md
│ │ ├── redis
│ │ │ ├── _index.md
│ │ │ └── redis.md
│ │ ├── spanner
│ │ │ ├── _index.md
│ │ │ ├── spanner-execute-sql.md
│ │ │ ├── spanner-list-tables.md
│ │ │ └── spanner-sql.md
│ │ ├── sqlite
│ │ │ ├── _index.md
│ │ │ ├── sqlite-execute-sql.md
│ │ │ └── sqlite-sql.md
│ │ ├── tidb
│ │ │ ├── _index.md
│ │ │ ├── tidb-execute-sql.md
│ │ │ └── tidb-sql.md
│ │ ├── trino
│ │ │ ├── _index.md
│ │ │ ├── trino-execute-sql.md
│ │ │ └── trino-sql.md
│ │ ├── utility
│ │ │ ├── _index.md
│ │ │ └── wait.md
│ │ ├── valkey
│ │ │ ├── _index.md
│ │ │ └── valkey.md
│ │ └── yuagbytedb
│ │ ├── _index.md
│ │ └── yugabytedb-sql.md
│ ├── samples
│ │ ├── _index.md
│ │ ├── alloydb
│ │ │ ├── _index.md
│ │ │ ├── ai-nl
│ │ │ │ ├── alloydb_ai_nl.ipynb
│ │ │ │ └── index.md
│ │ │ └── mcp_quickstart.md
│ │ ├── bigquery
│ │ │ ├── _index.md
│ │ │ ├── colab_quickstart_bigquery.ipynb
│ │ │ ├── local_quickstart.md
│ │ │ └── mcp_quickstart
│ │ │ ├── _index.md
│ │ │ ├── inspector_tools.png
│ │ │ └── inspector.png
│ │ └── looker
│ │ ├── _index.md
│ │ ├── looker_gemini_oauth
│ │ │ ├── _index.md
│ │ │ ├── authenticated.png
│ │ │ ├── authorize.png
│ │ │ └── registration.png
│ │ ├── looker_gemini.md
│ │ └── looker_mcp_inspector
│ │ ├── _index.md
│ │ ├── inspector_tools.png
│ │ └── inspector.png
│ └── sdks
│ ├── _index.md
│ ├── go-sdk.md
│ ├── js-sdk.md
│ └── python-sdk.md
├── gemini-extension.json
├── go.mod
├── go.sum
├── internal
│ ├── auth
│ │ ├── auth.go
│ │ └── google
│ │ └── google.go
│ ├── log
│ │ ├── handler.go
│ │ ├── log_test.go
│ │ ├── log.go
│ │ └── logger.go
│ ├── prebuiltconfigs
│ │ ├── prebuiltconfigs_test.go
│ │ ├── prebuiltconfigs.go
│ │ └── tools
│ │ ├── alloydb-postgres-admin.yaml
│ │ ├── alloydb-postgres-observability.yaml
│ │ ├── alloydb-postgres.yaml
│ │ ├── bigquery.yaml
│ │ ├── clickhouse.yaml
│ │ ├── cloud-sql-mssql-admin.yaml
│ │ ├── cloud-sql-mssql-observability.yaml
│ │ ├── cloud-sql-mssql.yaml
│ │ ├── cloud-sql-mysql-admin.yaml
│ │ ├── cloud-sql-mysql-observability.yaml
│ │ ├── cloud-sql-mysql.yaml
│ │ ├── cloud-sql-postgres-admin.yaml
│ │ ├── cloud-sql-postgres-observability.yaml
│ │ ├── cloud-sql-postgres.yaml
│ │ ├── dataplex.yaml
│ │ ├── firestore.yaml
│ │ ├── looker-conversational-analytics.yaml
│ │ ├── looker.yaml
│ │ ├── mssql.yaml
│ │ ├── mysql.yaml
│ │ ├── neo4j.yaml
│ │ ├── oceanbase.yaml
│ │ ├── postgres.yaml
│ │ ├── spanner-postgres.yaml
│ │ ├── spanner.yaml
│ │ └── sqlite.yaml
│ ├── server
│ │ ├── api_test.go
│ │ ├── api.go
│ │ ├── common_test.go
│ │ ├── config.go
│ │ ├── mcp
│ │ │ ├── jsonrpc
│ │ │ │ ├── jsonrpc_test.go
│ │ │ │ └── jsonrpc.go
│ │ │ ├── mcp.go
│ │ │ ├── util
│ │ │ │ └── lifecycle.go
│ │ │ ├── v20241105
│ │ │ │ ├── method.go
│ │ │ │ └── types.go
│ │ │ ├── v20250326
│ │ │ │ ├── method.go
│ │ │ │ └── types.go
│ │ │ └── v20250618
│ │ │ ├── method.go
│ │ │ └── types.go
│ │ ├── mcp_test.go
│ │ ├── mcp.go
│ │ ├── server_test.go
│ │ ├── server.go
│ │ ├── static
│ │ │ ├── assets
│ │ │ │ └── mcptoolboxlogo.png
│ │ │ ├── css
│ │ │ │ └── style.css
│ │ │ ├── index.html
│ │ │ ├── js
│ │ │ │ ├── auth.js
│ │ │ │ ├── loadTools.js
│ │ │ │ ├── mainContent.js
│ │ │ │ ├── navbar.js
│ │ │ │ ├── runTool.js
│ │ │ │ ├── toolDisplay.js
│ │ │ │ ├── tools.js
│ │ │ │ └── toolsets.js
│ │ │ ├── tools.html
│ │ │ └── toolsets.html
│ │ ├── web_test.go
│ │ └── web.go
│ ├── sources
│ │ ├── alloydbadmin
│ │ │ ├── alloydbadmin_test.go
│ │ │ └── alloydbadmin.go
│ │ ├── alloydbpg
│ │ │ ├── alloydb_pg_test.go
│ │ │ └── alloydb_pg.go
│ │ ├── bigquery
│ │ │ ├── bigquery_test.go
│ │ │ └── bigquery.go
│ │ ├── bigtable
│ │ │ ├── bigtable_test.go
│ │ │ └── bigtable.go
│ │ ├── cassandra
│ │ │ ├── cassandra_test.go
│ │ │ └── cassandra.go
│ │ ├── clickhouse
│ │ │ ├── clickhouse_test.go
│ │ │ └── clickhouse.go
│ │ ├── cloudmonitoring
│ │ │ ├── cloud_monitoring_test.go
│ │ │ └── cloud_monitoring.go
│ │ ├── cloudsqladmin
│ │ │ ├── cloud_sql_admin_test.go
│ │ │ └── cloud_sql_admin.go
│ │ ├── cloudsqlmssql
│ │ │ ├── cloud_sql_mssql_test.go
│ │ │ └── cloud_sql_mssql.go
│ │ ├── cloudsqlmysql
│ │ │ ├── cloud_sql_mysql_test.go
│ │ │ └── cloud_sql_mysql.go
│ │ ├── cloudsqlpg
│ │ │ ├── cloud_sql_pg_test.go
│ │ │ └── cloud_sql_pg.go
│ │ ├── couchbase
│ │ │ ├── couchbase_test.go
│ │ │ └── couchbase.go
│ │ ├── dataplex
│ │ │ ├── dataplex_test.go
│ │ │ └── dataplex.go
│ │ ├── dgraph
│ │ │ ├── dgraph_test.go
│ │ │ └── dgraph.go
│ │ ├── dialect.go
│ │ ├── firebird
│ │ │ ├── firebird_test.go
│ │ │ └── firebird.go
│ │ ├── firestore
│ │ │ ├── firestore_test.go
│ │ │ └── firestore.go
│ │ ├── http
│ │ │ ├── http_test.go
│ │ │ └── http.go
│ │ ├── ip_type.go
│ │ ├── looker
│ │ │ ├── looker_test.go
│ │ │ └── looker.go
│ │ ├── mongodb
│ │ │ ├── mongodb_test.go
│ │ │ └── mongodb.go
│ │ ├── mssql
│ │ │ ├── mssql_test.go
│ │ │ └── mssql.go
│ │ ├── mysql
│ │ │ ├── mysql_test.go
│ │ │ └── mysql.go
│ │ ├── neo4j
│ │ │ ├── neo4j_test.go
│ │ │ └── neo4j.go
│ │ ├── oceanbase
│ │ │ ├── oceanbase_test.go
│ │ │ └── oceanbase.go
│ │ ├── oracle
│ │ │ └── oracle.go
│ │ ├── postgres
│ │ │ ├── postgres_test.go
│ │ │ └── postgres.go
│ │ ├── redis
│ │ │ ├── redis_test.go
│ │ │ └── redis.go
│ │ ├── sources.go
│ │ ├── spanner
│ │ │ ├── spanner_test.go
│ │ │ └── spanner.go
│ │ ├── sqlite
│ │ │ ├── sqlite_test.go
│ │ │ └── sqlite.go
│ │ ├── tidb
│ │ │ ├── tidb_test.go
│ │ │ └── tidb.go
│ │ ├── trino
│ │ │ ├── trino_test.go
│ │ │ └── trino.go
│ │ ├── util.go
│ │ ├── valkey
│ │ │ ├── valkey_test.go
│ │ │ └── valkey.go
│ │ └── yugabytedb
│ │ ├── yugabytedb_test.go
│ │ └── yugabytedb.go
│ ├── telemetry
│ │ ├── instrumentation.go
│ │ └── telemetry.go
│ ├── testutils
│ │ └── testutils.go
│ ├── tools
│ │ ├── alloydb
│ │ │ ├── alloydbcreatecluster
│ │ │ │ ├── alloydbcreatecluster_test.go
│ │ │ │ └── alloydbcreatecluster.go
│ │ │ ├── alloydbcreateinstance
│ │ │ │ ├── alloydbcreateinstance_test.go
│ │ │ │ └── alloydbcreateinstance.go
│ │ │ ├── alloydbcreateuser
│ │ │ │ ├── alloydbcreateuser_test.go
│ │ │ │ └── alloydbcreateuser.go
│ │ │ ├── alloydbgetcluster
│ │ │ │ ├── alloydbgetcluster_test.go
│ │ │ │ └── alloydbgetcluster.go
│ │ │ ├── alloydbgetinstance
│ │ │ │ ├── alloydbgetinstance_test.go
│ │ │ │ └── alloydbgetinstance.go
│ │ │ ├── alloydbgetuser
│ │ │ │ ├── alloydbgetuser_test.go
│ │ │ │ └── alloydbgetuser.go
│ │ │ ├── alloydblistclusters
│ │ │ │ ├── alloydblistclusters_test.go
│ │ │ │ └── alloydblistclusters.go
│ │ │ ├── alloydblistinstances
│ │ │ │ ├── alloydblistinstances_test.go
│ │ │ │ └── alloydblistinstances.go
│ │ │ ├── alloydblistusers
│ │ │ │ ├── alloydblistusers_test.go
│ │ │ │ └── alloydblistusers.go
│ │ │ └── alloydbwaitforoperation
│ │ │ ├── alloydbwaitforoperation_test.go
│ │ │ └── alloydbwaitforoperation.go
│ │ ├── alloydbainl
│ │ │ ├── alloydbainl_test.go
│ │ │ └── alloydbainl.go
│ │ ├── bigquery
│ │ │ ├── bigqueryanalyzecontribution
│ │ │ │ ├── bigqueryanalyzecontribution_test.go
│ │ │ │ └── bigqueryanalyzecontribution.go
│ │ │ ├── bigquerycommon
│ │ │ │ ├── table_name_parser_test.go
│ │ │ │ ├── table_name_parser.go
│ │ │ │ └── util.go
│ │ │ ├── bigqueryconversationalanalytics
│ │ │ │ ├── bigqueryconversationalanalytics_test.go
│ │ │ │ └── bigqueryconversationalanalytics.go
│ │ │ ├── bigqueryexecutesql
│ │ │ │ ├── bigqueryexecutesql_test.go
│ │ │ │ └── bigqueryexecutesql.go
│ │ │ ├── bigqueryforecast
│ │ │ │ ├── bigqueryforecast_test.go
│ │ │ │ └── bigqueryforecast.go
│ │ │ ├── bigquerygetdatasetinfo
│ │ │ │ ├── bigquerygetdatasetinfo_test.go
│ │ │ │ └── bigquerygetdatasetinfo.go
│ │ │ ├── bigquerygettableinfo
│ │ │ │ ├── bigquerygettableinfo_test.go
│ │ │ │ └── bigquerygettableinfo.go
│ │ │ ├── bigquerylistdatasetids
│ │ │ │ ├── bigquerylistdatasetids_test.go
│ │ │ │ └── bigquerylistdatasetids.go
│ │ │ ├── bigquerylisttableids
│ │ │ │ ├── bigquerylisttableids_test.go
│ │ │ │ └── bigquerylisttableids.go
│ │ │ ├── bigquerysearchcatalog
│ │ │ │ ├── bigquerysearchcatalog_test.go
│ │ │ │ └── bigquerysearchcatalog.go
│ │ │ └── bigquerysql
│ │ │ ├── bigquerysql_test.go
│ │ │ └── bigquerysql.go
│ │ ├── bigtable
│ │ │ ├── bigtable_test.go
│ │ │ └── bigtable.go
│ │ ├── cassandra
│ │ │ └── cassandracql
│ │ │ ├── cassandracql_test.go
│ │ │ └── cassandracql.go
│ │ ├── clickhouse
│ │ │ ├── clickhouseexecutesql
│ │ │ │ ├── clickhouseexecutesql_test.go
│ │ │ │ └── clickhouseexecutesql.go
│ │ │ ├── clickhouselistdatabases
│ │ │ │ ├── clickhouselistdatabases_test.go
│ │ │ │ └── clickhouselistdatabases.go
│ │ │ ├── clickhouselisttables
│ │ │ │ ├── clickhouselisttables_test.go
│ │ │ │ └── clickhouselisttables.go
│ │ │ └── clickhousesql
│ │ │ ├── clickhousesql_test.go
│ │ │ └── clickhousesql.go
│ │ ├── cloudmonitoring
│ │ │ ├── cloudmonitoring_test.go
│ │ │ └── cloudmonitoring.go
│ │ ├── cloudsql
│ │ │ ├── cloudsqlcreatedatabase
│ │ │ │ ├── cloudsqlcreatedatabase_test.go
│ │ │ │ └── cloudsqlcreatedatabase.go
│ │ │ ├── cloudsqlcreateusers
│ │ │ │ ├── cloudsqlcreateusers_test.go
│ │ │ │ └── cloudsqlcreateusers.go
│ │ │ ├── cloudsqlgetinstances
│ │ │ │ ├── cloudsqlgetinstances_test.go
│ │ │ │ └── cloudsqlgetinstances.go
│ │ │ ├── cloudsqllistdatabases
│ │ │ │ ├── cloudsqllistdatabases_test.go
│ │ │ │ └── cloudsqllistdatabases.go
│ │ │ ├── cloudsqllistinstances
│ │ │ │ ├── cloudsqllistinstances_test.go
│ │ │ │ └── cloudsqllistinstances.go
│ │ │ └── cloudsqlwaitforoperation
│ │ │ ├── cloudsqlwaitforoperation_test.go
│ │ │ └── cloudsqlwaitforoperation.go
│ │ ├── cloudsqlmssql
│ │ │ └── cloudsqlmssqlcreateinstance
│ │ │ ├── cloudsqlmssqlcreateinstance_test.go
│ │ │ └── cloudsqlmssqlcreateinstance.go
│ │ ├── cloudsqlmysql
│ │ │ └── cloudsqlmysqlcreateinstance
│ │ │ ├── cloudsqlmysqlcreateinstance_test.go
│ │ │ └── cloudsqlmysqlcreateinstance.go
│ │ ├── cloudsqlpg
│ │ │ └── cloudsqlpgcreateinstances
│ │ │ ├── cloudsqlpgcreateinstances_test.go
│ │ │ └── cloudsqlpgcreateinstances.go
│ │ ├── common_test.go
│ │ ├── common.go
│ │ ├── couchbase
│ │ │ ├── couchbase_test.go
│ │ │ └── couchbase.go
│ │ ├── dataform
│ │ │ └── dataformcompilelocal
│ │ │ ├── dataformcompilelocal_test.go
│ │ │ └── dataformcompilelocal.go
│ │ ├── dataplex
│ │ │ ├── dataplexlookupentry
│ │ │ │ ├── dataplexlookupentry_test.go
│ │ │ │ └── dataplexlookupentry.go
│ │ │ ├── dataplexsearchaspecttypes
│ │ │ │ ├── dataplexsearchaspecttypes_test.go
│ │ │ │ └── dataplexsearchaspecttypes.go
│ │ │ └── dataplexsearchentries
│ │ │ ├── dataplexsearchentries_test.go
│ │ │ └── dataplexsearchentries.go
│ │ ├── dgraph
│ │ │ ├── dgraph_test.go
│ │ │ └── dgraph.go
│ │ ├── firebird
│ │ │ ├── firebirdexecutesql
│ │ │ │ ├── firebirdexecutesql_test.go
│ │ │ │ └── firebirdexecutesql.go
│ │ │ └── firebirdsql
│ │ │ ├── firebirdsql_test.go
│ │ │ └── firebirdsql.go
│ │ ├── firestore
│ │ │ ├── firestoreadddocuments
│ │ │ │ ├── firestoreadddocuments_test.go
│ │ │ │ └── firestoreadddocuments.go
│ │ │ ├── firestoredeletedocuments
│ │ │ │ ├── firestoredeletedocuments_test.go
│ │ │ │ └── firestoredeletedocuments.go
│ │ │ ├── firestoregetdocuments
│ │ │ │ ├── firestoregetdocuments_test.go
│ │ │ │ └── firestoregetdocuments.go
│ │ │ ├── firestoregetrules
│ │ │ │ ├── firestoregetrules_test.go
│ │ │ │ └── firestoregetrules.go
│ │ │ ├── firestorelistcollections
│ │ │ │ ├── firestorelistcollections_test.go
│ │ │ │ └── firestorelistcollections.go
│ │ │ ├── firestorequery
│ │ │ │ ├── firestorequery_test.go
│ │ │ │ └── firestorequery.go
│ │ │ ├── firestorequerycollection
│ │ │ │ ├── firestorequerycollection_test.go
│ │ │ │ └── firestorequerycollection.go
│ │ │ ├── firestoreupdatedocument
│ │ │ │ ├── firestoreupdatedocument_test.go
│ │ │ │ └── firestoreupdatedocument.go
│ │ │ ├── firestorevalidaterules
│ │ │ │ ├── firestorevalidaterules_test.go
│ │ │ │ └── firestorevalidaterules.go
│ │ │ └── util
│ │ │ ├── converter_test.go
│ │ │ ├── converter.go
│ │ │ ├── validator_test.go
│ │ │ └── validator.go
│ │ ├── http
│ │ │ ├── http_test.go
│ │ │ └── http.go
│ │ ├── http_method.go
│ │ ├── looker
│ │ │ ├── lookeradddashboardelement
│ │ │ │ ├── lookeradddashboardelement_test.go
│ │ │ │ └── lookeradddashboardelement.go
│ │ │ ├── lookercommon
│ │ │ │ ├── lookercommon_test.go
│ │ │ │ └── lookercommon.go
│ │ │ ├── lookerconversationalanalytics
│ │ │ │ ├── lookerconversationalanalytics_test.go
│ │ │ │ └── lookerconversationalanalytics.go
│ │ │ ├── lookergetdashboards
│ │ │ │ ├── lookergetdashboards_test.go
│ │ │ │ └── lookergetdashboards.go
│ │ │ ├── lookergetdimensions
│ │ │ │ ├── lookergetdimensions_test.go
│ │ │ │ └── lookergetdimensions.go
│ │ │ ├── lookergetexplores
│ │ │ │ ├── lookergetexplores_test.go
│ │ │ │ └── lookergetexplores.go
│ │ │ ├── lookergetfilters
│ │ │ │ ├── lookergetfilters_test.go
│ │ │ │ └── lookergetfilters.go
│ │ │ ├── lookergetlooks
│ │ │ │ ├── lookergetlooks_test.go
│ │ │ │ └── lookergetlooks.go
│ │ │ ├── lookergetmeasures
│ │ │ │ ├── lookergetmeasures_test.go
│ │ │ │ └── lookergetmeasures.go
│ │ │ ├── lookergetmodels
│ │ │ │ ├── lookergetmodels_test.go
│ │ │ │ └── lookergetmodels.go
│ │ │ ├── lookergetparameters
│ │ │ │ ├── lookergetparameters_test.go
│ │ │ │ └── lookergetparameters.go
│ │ │ ├── lookerhealthanalyze
│ │ │ │ ├── lookerhealthanalyze_test.go
│ │ │ │ └── lookerhealthanalyze.go
│ │ │ ├── lookerhealthpulse
│ │ │ │ ├── lookerhealthpulse_test.go
│ │ │ │ └── lookerhealthpulse.go
│ │ │ ├── lookerhealthvacuum
│ │ │ │ ├── lookerhealthvacuum_test.go
│ │ │ │ └── lookerhealthvacuum.go
│ │ │ ├── lookermakedashboard
│ │ │ │ ├── lookermakedashboard_test.go
│ │ │ │ └── lookermakedashboard.go
│ │ │ ├── lookermakelook
│ │ │ │ ├── lookermakelook_test.go
│ │ │ │ └── lookermakelook.go
│ │ │ ├── lookerquery
│ │ │ │ ├── lookerquery_test.go
│ │ │ │ └── lookerquery.go
│ │ │ ├── lookerquerysql
│ │ │ │ ├── lookerquerysql_test.go
│ │ │ │ └── lookerquerysql.go
│ │ │ ├── lookerqueryurl
│ │ │ │ ├── lookerqueryurl_test.go
│ │ │ │ └── lookerqueryurl.go
│ │ │ └── lookerrunlook
│ │ │ ├── lookerrunlook_test.go
│ │ │ └── lookerrunlook.go
│ │ ├── mongodb
│ │ │ ├── mongodbaggregate
│ │ │ │ ├── mongodbaggregate_test.go
│ │ │ │ └── mongodbaggregate.go
│ │ │ ├── mongodbdeletemany
│ │ │ │ ├── mongodbdeletemany_test.go
│ │ │ │ └── mongodbdeletemany.go
│ │ │ ├── mongodbdeleteone
│ │ │ │ ├── mongodbdeleteone_test.go
│ │ │ │ └── mongodbdeleteone.go
│ │ │ ├── mongodbfind
│ │ │ │ ├── mongodbfind_test.go
│ │ │ │ └── mongodbfind.go
│ │ │ ├── mongodbfindone
│ │ │ │ ├── mongodbfindone_test.go
│ │ │ │ └── mongodbfindone.go
│ │ │ ├── mongodbinsertmany
│ │ │ │ ├── mongodbinsertmany_test.go
│ │ │ │ └── mongodbinsertmany.go
│ │ │ ├── mongodbinsertone
│ │ │ │ ├── mongodbinsertone_test.go
│ │ │ │ └── mongodbinsertone.go
│ │ │ ├── mongodbupdatemany
│ │ │ │ ├── mongodbupdatemany_test.go
│ │ │ │ └── mongodbupdatemany.go
│ │ │ └── mongodbupdateone
│ │ │ ├── mongodbupdateone_test.go
│ │ │ └── mongodbupdateone.go
│ │ ├── mssql
│ │ │ ├── mssqlexecutesql
│ │ │ │ ├── mssqlexecutesql_test.go
│ │ │ │ └── mssqlexecutesql.go
│ │ │ ├── mssqllisttables
│ │ │ │ ├── mssqllisttables_test.go
│ │ │ │ └── mssqllisttables.go
│ │ │ └── mssqlsql
│ │ │ ├── mssqlsql_test.go
│ │ │ └── mssqlsql.go
│ │ ├── mysql
│ │ │ ├── mysqlcommon
│ │ │ │ └── mysqlcommon.go
│ │ │ ├── mysqlexecutesql
│ │ │ │ ├── mysqlexecutesql_test.go
│ │ │ │ └── mysqlexecutesql.go
│ │ │ ├── mysqllistactivequeries
│ │ │ │ ├── mysqllistactivequeries_test.go
│ │ │ │ └── mysqllistactivequeries.go
│ │ │ ├── mysqllisttablefragmentation
│ │ │ │ ├── mysqllisttablefragmentation_test.go
│ │ │ │ └── mysqllisttablefragmentation.go
│ │ │ ├── mysqllisttables
│ │ │ │ ├── mysqllisttables_test.go
│ │ │ │ └── mysqllisttables.go
│ │ │ ├── mysqllisttablesmissinguniqueindexes
│ │ │ │ ├── mysqllisttablesmissinguniqueindexes_test.go
│ │ │ │ └── mysqllisttablesmissinguniqueindexes.go
│ │ │ └── mysqlsql
│ │ │ ├── mysqlsql_test.go
│ │ │ └── mysqlsql.go
│ │ ├── neo4j
│ │ │ ├── neo4jcypher
│ │ │ │ ├── neo4jcypher_test.go
│ │ │ │ └── neo4jcypher.go
│ │ │ ├── neo4jexecutecypher
│ │ │ │ ├── classifier
│ │ │ │ │ ├── classifier_test.go
│ │ │ │ │ └── classifier.go
│ │ │ │ ├── neo4jexecutecypher_test.go
│ │ │ │ └── neo4jexecutecypher.go
│ │ │ └── neo4jschema
│ │ │ ├── cache
│ │ │ │ ├── cache_test.go
│ │ │ │ └── cache.go
│ │ │ ├── helpers
│ │ │ │ ├── helpers_test.go
│ │ │ │ └── helpers.go
│ │ │ ├── neo4jschema_test.go
│ │ │ ├── neo4jschema.go
│ │ │ └── types
│ │ │ └── types.go
│ │ ├── oceanbase
│ │ │ ├── oceanbaseexecutesql
│ │ │ │ ├── oceanbaseexecutesql_test.go
│ │ │ │ └── oceanbaseexecutesql.go
│ │ │ └── oceanbasesql
│ │ │ ├── oceanbasesql_test.go
│ │ │ └── oceanbasesql.go
│ │ ├── oracle
│ │ │ ├── oracleexecutesql
│ │ │ │ └── oracleexecutesql.go
│ │ │ └── oraclesql
│ │ │ └── oraclesql.go
│ │ ├── parameters_test.go
│ │ ├── parameters.go
│ │ ├── postgres
│ │ │ ├── postgresexecutesql
│ │ │ │ ├── postgresexecutesql_test.go
│ │ │ │ └── postgresexecutesql.go
│ │ │ ├── postgreslistactivequeries
│ │ │ │ ├── postgreslistactivequeries_test.go
│ │ │ │ └── postgreslistactivequeries.go
│ │ │ ├── postgreslistavailableextensions
│ │ │ │ ├── postgreslistavailableextensions_test.go
│ │ │ │ └── postgreslistavailableextensions.go
│ │ │ ├── postgreslistinstalledextensions
│ │ │ │ ├── postgreslistinstalledextensions_test.go
│ │ │ │ └── postgreslistinstalledextensions.go
│ │ │ ├── postgreslisttables
│ │ │ │ ├── postgreslisttables_test.go
│ │ │ │ └── postgreslisttables.go
│ │ │ └── postgressql
│ │ │ ├── postgressql_test.go
│ │ │ └── postgressql.go
│ │ ├── redis
│ │ │ ├── redis_test.go
│ │ │ └── redis.go
│ │ ├── spanner
│ │ │ ├── spannerexecutesql
│ │ │ │ ├── spannerexecutesql_test.go
│ │ │ │ └── spannerexecutesql.go
│ │ │ ├── spannerlisttables
│ │ │ │ ├── spannerlisttables_test.go
│ │ │ │ └── spannerlisttables.go
│ │ │ └── spannersql
│ │ │ ├── spanner_test.go
│ │ │ └── spannersql.go
│ │ ├── sqlite
│ │ │ ├── sqliteexecutesql
│ │ │ │ ├── sqliteexecutesql_test.go
│ │ │ │ └── sqliteexecutesql.go
│ │ │ └── sqlitesql
│ │ │ ├── sqlitesql_test.go
│ │ │ └── sqlitesql.go
│ │ ├── tidb
│ │ │ ├── tidbexecutesql
│ │ │ │ ├── tidbexecutesql_test.go
│ │ │ │ └── tidbexecutesql.go
│ │ │ └── tidbsql
│ │ │ ├── tidbsql_test.go
│ │ │ └── tidbsql.go
│ │ ├── tools_test.go
│ │ ├── tools.go
│ │ ├── toolsets.go
│ │ ├── trino
│ │ │ ├── trinoexecutesql
│ │ │ │ ├── trinoexecutesql_test.go
│ │ │ │ └── trinoexecutesql.go
│ │ │ └── trinosql
│ │ │ ├── trinosql_test.go
│ │ │ └── trinosql.go
│ │ ├── utility
│ │ │ └── wait
│ │ │ ├── wait_test.go
│ │ │ └── wait.go
│ │ ├── valkey
│ │ │ ├── valkey_test.go
│ │ │ └── valkey.go
│ │ └── yugabytedbsql
│ │ ├── yugabytedbsql_test.go
│ │ └── yugabytedbsql.go
│ └── util
│ └── util.go
├── LICENSE
├── logo.png
├── main.go
├── MCP-TOOLBOX-EXTENSION.md
├── README.md
└── tests
├── alloydb
│ ├── alloydb_integration_test.go
│ └── alloydb_wait_for_operation_test.go
├── alloydbainl
│ └── alloydb_ai_nl_integration_test.go
├── alloydbpg
│ └── alloydb_pg_integration_test.go
├── auth.go
├── bigquery
│ └── bigquery_integration_test.go
├── bigtable
│ └── bigtable_integration_test.go
├── cassandra
│ └── cassandra_integration_test.go
├── clickhouse
│ └── clickhouse_integration_test.go
├── cloudmonitoring
│ └── cloud_monitoring_integration_test.go
├── cloudsql
│ ├── cloud_sql_create_database_test.go
│ ├── cloud_sql_create_users_test.go
│ ├── cloud_sql_get_instances_test.go
│ ├── cloud_sql_list_databases_test.go
│ ├── cloudsql_list_instances_test.go
│ └── cloudsql_wait_for_operation_test.go
├── cloudsqlmssql
│ ├── cloud_sql_mssql_create_instance_integration_test.go
│ └── cloud_sql_mssql_integration_test.go
├── cloudsqlmysql
│ ├── cloud_sql_mysql_create_instance_integration_test.go
│ └── cloud_sql_mysql_integration_test.go
├── cloudsqlpg
│ ├── cloud_sql_pg_create_instances_test.go
│ └── cloud_sql_pg_integration_test.go
├── common.go
├── couchbase
│ └── couchbase_integration_test.go
├── dataform
│ └── dataform_integration_test.go
├── dataplex
│ └── dataplex_integration_test.go
├── dgraph
│ └── dgraph_integration_test.go
├── firebird
│ └── firebird_integration_test.go
├── firestore
│ └── firestore_integration_test.go
├── http
│ └── http_integration_test.go
├── looker
│ └── looker_integration_test.go
├── mongodb
│ └── mongodb_integration_test.go
├── mssql
│ └── mssql_integration_test.go
├── mysql
│ └── mysql_integration_test.go
├── neo4j
│ └── neo4j_integration_test.go
├── oceanbase
│ └── oceanbase_integration_test.go
├── option.go
├── oracle
│ └── oracle_integration_test.go
├── postgres
│ └── postgres_integration_test.go
├── redis
│ └── redis_test.go
├── server.go
├── source.go
├── spanner
│ └── spanner_integration_test.go
├── sqlite
│ └── sqlite_integration_test.go
├── tidb
│ └── tidb_integration_test.go
├── tool.go
├── trino
│ └── trino_integration_test.go
├── utility
│ └── wait_integration_test.go
├── valkey
│ └── valkey_test.go
└── yugabytedb
└── yugabytedb_integration_test.go
```
# Files
--------------------------------------------------------------------------------
/tests/dataplex/dataplex_integration_test.go:
--------------------------------------------------------------------------------
```go
1 | // Copyright 2025 Google LLC
2 | //
3 | // Licensed under the Apache License, Version 2.0 (the "License");
4 | // you may not use this file except in compliance with the License.
5 | // You may obtain a copy of the License at
6 | //
7 | // http://www.apache.org/licenses/LICENSE-2.0
8 | //
9 | // Unless required by applicable law or agreed to in writing, software
10 | // distributed under the License is distributed on an "AS IS" BASIS,
11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | // See the License for the specific language governing permissions and
13 | // limitations under the License.
14 |
15 | package dataplex
16 |
17 | import (
18 | "bytes"
19 | "context"
20 | "encoding/json"
21 | "fmt"
22 | "io"
23 | "net/http"
24 | "os"
25 | "regexp"
26 | "strings"
27 | "testing"
28 | "time"
29 |
30 | bigqueryapi "cloud.google.com/go/bigquery"
31 | dataplex "cloud.google.com/go/dataplex/apiv1"
32 | dataplexpb "cloud.google.com/go/dataplex/apiv1/dataplexpb"
33 | "github.com/google/uuid"
34 | "github.com/googleapis/genai-toolbox/internal/testutils"
35 | "github.com/googleapis/genai-toolbox/tests"
36 | "golang.org/x/oauth2/google"
37 | "google.golang.org/api/googleapi"
38 | "google.golang.org/api/iterator"
39 | "google.golang.org/api/option"
40 | )
41 |
42 | var (
43 | DataplexSourceKind = "dataplex"
44 | DataplexSearchEntriesToolKind = "dataplex-search-entries"
45 | DataplexLookupEntryToolKind = "dataplex-lookup-entry"
46 | DataplexSearchAspectTypesToolKind = "dataplex-search-aspect-types"
47 | DataplexProject = os.Getenv("DATAPLEX_PROJECT")
48 | )
49 |
50 | func getDataplexVars(t *testing.T) map[string]any {
51 | switch "" {
52 | case DataplexProject:
53 | t.Fatal("'DATAPLEX_PROJECT' not set")
54 | }
55 | return map[string]any{
56 | "kind": DataplexSourceKind,
57 | "project": DataplexProject,
58 | }
59 | }
60 |
61 | // Copied over from bigquery.go
62 | func initBigQueryConnection(ctx context.Context, project string) (*bigqueryapi.Client, error) {
63 | cred, err := google.FindDefaultCredentials(ctx, bigqueryapi.Scope)
64 | if err != nil {
65 | return nil, fmt.Errorf("failed to find default Google Cloud credentials with scope %q: %w", bigqueryapi.Scope, err)
66 | }
67 |
68 | client, err := bigqueryapi.NewClient(ctx, project, option.WithCredentials(cred))
69 | if err != nil {
70 | return nil, fmt.Errorf("failed to create BigQuery client for project %q: %w", project, err)
71 | }
72 | return client, nil
73 | }
74 |
75 | func initDataplexConnection(ctx context.Context) (*dataplex.CatalogClient, error) {
76 | cred, err := google.FindDefaultCredentials(ctx)
77 | if err != nil {
78 | return nil, fmt.Errorf("failed to find default Google Cloud credentials: %w", err)
79 | }
80 |
81 | client, err := dataplex.NewCatalogClient(ctx, option.WithCredentials(cred))
82 | if err != nil {
83 | return nil, fmt.Errorf("failed to create Dataplex client %w", err)
84 | }
85 | return client, nil
86 | }
87 |
88 | func TestDataplexToolEndpoints(t *testing.T) {
89 | sourceConfig := getDataplexVars(t)
90 | ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
91 | defer cancel()
92 |
93 | var args []string
94 |
95 | bigqueryClient, err := initBigQueryConnection(ctx, DataplexProject)
96 | if err != nil {
97 | t.Fatalf("unable to create Cloud SQL connection pool: %s", err)
98 | }
99 |
100 | dataplexClient, err := initDataplexConnection(ctx)
101 | if err != nil {
102 | t.Fatalf("unable to create Dataplex connection: %s", err)
103 | }
104 |
105 | // create resources with UUID
106 | datasetName := fmt.Sprintf("temp_toolbox_test_%s", strings.ReplaceAll(uuid.New().String(), "-", ""))
107 | tableName := fmt.Sprintf("param_table_%s", strings.ReplaceAll(uuid.New().String(), "-", ""))
108 | aspectTypeId := fmt.Sprintf("param-aspect-type-%s", strings.ReplaceAll(uuid.New().String(), "-", ""))
109 |
110 | teardownTable1 := setupBigQueryTable(t, ctx, bigqueryClient, datasetName, tableName)
111 | teardownAspectType1 := setupDataplexThirdPartyAspectType(t, ctx, dataplexClient, aspectTypeId)
112 | time.Sleep(2 * time.Minute) // wait for table and aspect type to be ingested
113 | defer teardownTable1(t)
114 | defer teardownAspectType1(t)
115 |
116 | toolsFile := getDataplexToolsConfig(sourceConfig)
117 |
118 | cmd, cleanup, err := tests.StartCmd(ctx, toolsFile, args...)
119 | if err != nil {
120 | t.Fatalf("command initialization returned an error: %s", err)
121 | }
122 | defer cleanup()
123 |
124 | waitCtx, cancel := context.WithTimeout(ctx, 3*time.Minute)
125 | defer cancel()
126 | out, err := testutils.WaitForString(waitCtx, regexp.MustCompile(`Server ready to serve`), cmd.Out)
127 | if err != nil {
128 | t.Logf("toolbox command logs: \n%s", out)
129 | t.Fatalf("toolbox didn't start successfully: %s", err)
130 | }
131 |
132 | runDataplexToolGetTest(t)
133 | runDataplexSearchEntriesToolInvokeTest(t, tableName, datasetName)
134 | runDataplexLookupEntryToolInvokeTest(t, tableName, datasetName)
135 | runDataplexSearchAspectTypesToolInvokeTest(t, aspectTypeId)
136 | }
137 |
138 | func setupBigQueryTable(t *testing.T, ctx context.Context, client *bigqueryapi.Client, datasetName string, tableName string) func(*testing.T) {
139 | // Create dataset
140 | dataset := client.Dataset(datasetName)
141 | _, err := dataset.Metadata(ctx)
142 |
143 | if err != nil {
144 | apiErr, ok := err.(*googleapi.Error)
145 | if !ok || apiErr.Code != 404 {
146 | t.Fatalf("Failed to check dataset %q existence: %v", datasetName, err)
147 | }
148 | metadataToCreate := &bigqueryapi.DatasetMetadata{Name: datasetName}
149 | if err := dataset.Create(ctx, metadataToCreate); err != nil {
150 | t.Fatalf("Failed to create dataset %q: %v", datasetName, err)
151 | }
152 | }
153 |
154 | // Create table
155 | tab := client.Dataset(datasetName).Table(tableName)
156 | meta := &bigqueryapi.TableMetadata{}
157 | if err := tab.Create(ctx, meta); err != nil {
158 | t.Fatalf("Create table job for %s failed: %v", tableName, err)
159 | }
160 |
161 | return func(t *testing.T) {
162 | // tear down table
163 | dropSQL := fmt.Sprintf("drop table %s.%s", datasetName, tableName)
164 | dropJob, err := client.Query(dropSQL).Run(ctx)
165 | if err != nil {
166 | t.Errorf("Failed to start drop table job for %s: %v", tableName, err)
167 | return
168 | }
169 | dropStatus, err := dropJob.Wait(ctx)
170 | if err != nil {
171 | t.Errorf("Failed to wait for drop table job for %s: %v", tableName, err)
172 | return
173 | }
174 | if err := dropStatus.Err(); err != nil {
175 | t.Errorf("Error dropping table %s: %v", tableName, err)
176 | }
177 |
178 | // tear down dataset
179 | datasetToTeardown := client.Dataset(datasetName)
180 | tablesIterator := datasetToTeardown.Tables(ctx)
181 | _, err = tablesIterator.Next()
182 |
183 | if err == iterator.Done {
184 | if err := datasetToTeardown.Delete(ctx); err != nil {
185 | t.Errorf("Failed to delete dataset %s: %v", datasetName, err)
186 | }
187 | } else if err != nil {
188 | t.Errorf("Failed to list tables in dataset %s to check emptiness: %v.", datasetName, err)
189 | }
190 | }
191 | }
192 |
193 | func setupDataplexThirdPartyAspectType(t *testing.T, ctx context.Context, client *dataplex.CatalogClient, aspectTypeId string) func(*testing.T) {
194 | parent := fmt.Sprintf("projects/%s/locations/us", DataplexProject)
195 | createAspectTypeReq := &dataplexpb.CreateAspectTypeRequest{
196 | Parent: parent,
197 | AspectTypeId: aspectTypeId,
198 | AspectType: &dataplexpb.AspectType{
199 | Name: fmt.Sprintf("%s/aspectTypes/%s", parent, aspectTypeId),
200 | MetadataTemplate: &dataplexpb.AspectType_MetadataTemplate{
201 | Name: "UserSchema",
202 | Type: "record",
203 | },
204 | },
205 | }
206 | _, err := client.CreateAspectType(ctx, createAspectTypeReq)
207 | if err != nil {
208 | t.Fatalf("Failed to create aspect type %s: %v", aspectTypeId, err)
209 | }
210 |
211 | return func(t *testing.T) {
212 | // tear down aspect type
213 | deleteAspectTypeReq := &dataplexpb.DeleteAspectTypeRequest{
214 | Name: fmt.Sprintf("%s/aspectTypes/%s", parent, aspectTypeId),
215 | }
216 | if _, err := client.DeleteAspectType(ctx, deleteAspectTypeReq); err != nil {
217 | t.Errorf("Failed to delete aspect type %s: %v", aspectTypeId, err)
218 | }
219 | }
220 | }
221 |
222 | func getDataplexToolsConfig(sourceConfig map[string]any) map[string]any {
223 | // Write config into a file and pass it to command
224 | toolsFile := map[string]any{
225 | "sources": map[string]any{
226 | "my-dataplex-instance": sourceConfig,
227 | },
228 | "authServices": map[string]any{
229 | "my-google-auth": map[string]any{
230 | "kind": "google",
231 | "clientId": tests.ClientId,
232 | },
233 | },
234 | "tools": map[string]any{
235 | "my-dataplex-search-entries-tool": map[string]any{
236 | "kind": DataplexSearchEntriesToolKind,
237 | "source": "my-dataplex-instance",
238 | "description": "Simple dataplex search entries tool to test end to end functionality.",
239 | },
240 | "my-auth-dataplex-search-entries-tool": map[string]any{
241 | "kind": DataplexSearchEntriesToolKind,
242 | "source": "my-dataplex-instance",
243 | "description": "Simple dataplex search entries tool to test end to end functionality.",
244 | "authRequired": []string{"my-google-auth"},
245 | },
246 | "my-dataplex-lookup-entry-tool": map[string]any{
247 | "kind": DataplexLookupEntryToolKind,
248 | "source": "my-dataplex-instance",
249 | "description": "Simple dataplex lookup entry tool to test end to end functionality.",
250 | },
251 | "my-auth-dataplex-lookup-entry-tool": map[string]any{
252 | "kind": DataplexLookupEntryToolKind,
253 | "source": "my-dataplex-instance",
254 | "description": "Simple dataplex lookup entry tool to test end to end functionality.",
255 | "authRequired": []string{"my-google-auth"},
256 | },
257 | "my-dataplex-search-aspect-types-tool": map[string]any{
258 | "kind": DataplexSearchAspectTypesToolKind,
259 | "source": "my-dataplex-instance",
260 | "description": "Simple dataplex search aspect types tool to test end to end functionality.",
261 | },
262 | "my-auth-dataplex-search-aspect-types-tool": map[string]any{
263 | "kind": DataplexSearchAspectTypesToolKind,
264 | "source": "my-dataplex-instance",
265 | "description": "Simple dataplex search aspect types tool to test end to end functionality.",
266 | "authRequired": []string{"my-google-auth"},
267 | },
268 | },
269 | }
270 |
271 | return toolsFile
272 | }
273 |
274 | func runDataplexToolGetTest(t *testing.T) {
275 | testCases := []struct {
276 | name string
277 | toolName string
278 | expectedParams []string
279 | }{
280 | {
281 | name: "get my-dataplex-search-entries-tool",
282 | toolName: "my-dataplex-search-entries-tool",
283 | expectedParams: []string{"pageSize", "query", "orderBy"},
284 | },
285 | {
286 | name: "get my-dataplex-lookup-entry-tool",
287 | toolName: "my-dataplex-lookup-entry-tool",
288 | expectedParams: []string{"name", "view", "aspectTypes", "entry"},
289 | },
290 | {
291 | name: "get my-dataplex-search-aspect-types-tool",
292 | toolName: "my-dataplex-search-aspect-types-tool",
293 | expectedParams: []string{"pageSize", "query", "orderBy"},
294 | },
295 | }
296 |
297 | for _, tc := range testCases {
298 | t.Run(tc.name, func(t *testing.T) {
299 | resp, err := http.Get(fmt.Sprintf("http://127.0.0.1:5000/api/tool/%s/", tc.toolName))
300 | if err != nil {
301 | t.Fatalf("error when sending a request: %s", err)
302 | }
303 | defer resp.Body.Close()
304 | if resp.StatusCode != 200 {
305 | t.Fatalf("response status code is not 200")
306 | }
307 | var body map[string]interface{}
308 | err = json.NewDecoder(resp.Body).Decode(&body)
309 | if err != nil {
310 | t.Fatalf("error parsing response body")
311 | }
312 | got, ok := body["tools"]
313 | if !ok {
314 | t.Fatalf("unable to find tools in response body")
315 | }
316 |
317 | toolsMap, ok := got.(map[string]interface{})
318 | if !ok {
319 | t.Fatalf("expected 'tools' to be a map, got %T", got)
320 | }
321 | tool, ok := toolsMap[tc.toolName].(map[string]interface{})
322 | if !ok {
323 | t.Fatalf("expected tool %q to be a map, got %T", tc.toolName, toolsMap[tc.toolName])
324 | }
325 | params, ok := tool["parameters"].([]interface{})
326 | if !ok {
327 | t.Fatalf("expected 'parameters' to be a slice, got %T", tool["parameters"])
328 | }
329 | paramSet := make(map[string]struct{})
330 | for _, param := range params {
331 | paramMap, ok := param.(map[string]interface{})
332 | if ok {
333 | if name, ok := paramMap["name"].(string); ok {
334 | paramSet[name] = struct{}{}
335 | }
336 | }
337 | }
338 | var missing []string
339 | for _, want := range tc.expectedParams {
340 | if _, found := paramSet[want]; !found {
341 | missing = append(missing, want)
342 | }
343 | }
344 | if len(missing) > 0 {
345 | t.Fatalf("missing parameters for tool %q: %v", tc.toolName, missing)
346 | }
347 | })
348 | }
349 | }
350 |
351 | func runDataplexSearchEntriesToolInvokeTest(t *testing.T, tableName string, datasetName string) {
352 | idToken, err := tests.GetGoogleIdToken(tests.ClientId)
353 | if err != nil {
354 | t.Fatalf("error getting Google ID token: %s", err)
355 | }
356 |
357 | testCases := []struct {
358 | name string
359 | api string
360 | requestHeader map[string]string
361 | requestBody io.Reader
362 | wantStatusCode int
363 | expectResult bool
364 | wantContentKey string
365 | }{
366 | {
367 | name: "Success - Entry Found",
368 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-search-entries-tool/invoke",
369 | requestHeader: map[string]string{},
370 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"displayname=%s system=bigquery parent:%s\"}", tableName, datasetName))),
371 | wantStatusCode: 200,
372 | expectResult: true,
373 | wantContentKey: "dataplex_entry",
374 | },
375 | {
376 | name: "Success with Authorization - Entry Found",
377 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-entries-tool/invoke",
378 | requestHeader: map[string]string{"my-google-auth_token": idToken},
379 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"displayname=%s system=bigquery parent:%s\"}", tableName, datasetName))),
380 | wantStatusCode: 200,
381 | expectResult: true,
382 | wantContentKey: "dataplex_entry",
383 | },
384 | {
385 | name: "Failure - Invalid Authorization Token",
386 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-entries-tool/invoke",
387 | requestHeader: map[string]string{"my-google-auth_token": "invalid_token"},
388 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"displayname=%s system=bigquery parent:%s\"}", tableName, datasetName))),
389 | wantStatusCode: 401,
390 | expectResult: false,
391 | wantContentKey: "dataplex_entry",
392 | },
393 | {
394 | name: "Failure - Without Authorization Token",
395 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-entries-tool/invoke",
396 | requestHeader: map[string]string{},
397 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"displayname=%s system=bigquery parent:%s\"}", tableName, datasetName))),
398 | wantStatusCode: 401,
399 | expectResult: false,
400 | wantContentKey: "dataplex_entry",
401 | },
402 | {
403 | name: "Failure - Entry Not Found",
404 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-search-entries-tool/invoke",
405 | requestHeader: map[string]string{},
406 | requestBody: bytes.NewBuffer([]byte(`{"query":"displayname=\"\" system=bigquery parent:\"\""}`)),
407 | wantStatusCode: 200,
408 | expectResult: false,
409 | wantContentKey: "",
410 | },
411 | }
412 |
413 | for _, tc := range testCases {
414 | t.Run(tc.name, func(t *testing.T) {
415 | req, err := http.NewRequest(http.MethodPost, tc.api, tc.requestBody)
416 | if err != nil {
417 | t.Fatalf("unable to create request: %s", err)
418 | }
419 | req.Header.Add("Content-type", "application/json")
420 | for k, v := range tc.requestHeader {
421 | req.Header.Add(k, v)
422 | }
423 | resp, err := http.DefaultClient.Do(req)
424 | if err != nil {
425 | t.Fatalf("unable to send request: %s", err)
426 | }
427 | defer resp.Body.Close()
428 | if resp.StatusCode != tc.wantStatusCode {
429 | t.Fatalf("response status code is not %d. It is %d", tc.wantStatusCode, resp.StatusCode)
430 | bodyBytes, _ := io.ReadAll(resp.Body)
431 | t.Fatalf("Response body: %s", string(bodyBytes))
432 | }
433 | var result map[string]interface{}
434 | if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
435 | t.Fatalf("error parsing response body: %s", err)
436 | }
437 | resultStr, ok := result["result"].(string)
438 | if !ok {
439 | if result["result"] == nil && !tc.expectResult {
440 | return
441 | }
442 | t.Fatalf("expected 'result' field to be a string, got %T", result["result"])
443 | }
444 | if !tc.expectResult && (resultStr == "" || resultStr == "[]") {
445 | return
446 | }
447 | var entries []interface{}
448 | if err := json.Unmarshal([]byte(resultStr), &entries); err != nil {
449 | t.Fatalf("error unmarshalling result string: %v", err)
450 | }
451 |
452 | if tc.expectResult {
453 | if len(entries) != 1 {
454 | t.Fatalf("expected exactly one entry, but got %d", len(entries))
455 | }
456 | entry, ok := entries[0].(map[string]interface{})
457 | if !ok {
458 | t.Fatalf("expected first entry to be a map, got %T", entries[0])
459 | }
460 | if _, ok := entry[tc.wantContentKey]; !ok {
461 | t.Fatalf("expected entry to have key '%s', but it was not found in %v", tc.wantContentKey, entry)
462 | }
463 | } else {
464 | if len(entries) != 0 {
465 | t.Fatalf("expected 0 entries, but got %d", len(entries))
466 | }
467 | }
468 | })
469 | }
470 | }
471 |
472 | func runDataplexLookupEntryToolInvokeTest(t *testing.T, tableName string, datasetName string) {
473 | idToken, err := tests.GetGoogleIdToken(tests.ClientId)
474 | if err != nil {
475 | t.Fatalf("error getting Google ID token: %s", err)
476 | }
477 |
478 | testCases := []struct {
479 | name string
480 | wantStatusCode int
481 | api string
482 | requestHeader map[string]string
483 | requestBody io.Reader
484 | expectResult bool
485 | wantContentKey string
486 | dontWantContentKey string
487 | aspectCheck bool
488 | reqBodyMap map[string]any
489 | }{
490 | {
491 | name: "Success - Entry Found",
492 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-lookup-entry-tool/invoke",
493 | requestHeader: map[string]string{},
494 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s\"}", DataplexProject, DataplexProject, DataplexProject, datasetName))),
495 | wantStatusCode: 200,
496 | expectResult: true,
497 | wantContentKey: "name",
498 | },
499 | {
500 | name: "Success - Entry Found with Authorization",
501 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-lookup-entry-tool/invoke",
502 | requestHeader: map[string]string{"my-google-auth_token": idToken},
503 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s\"}", DataplexProject, DataplexProject, DataplexProject, datasetName))),
504 | wantStatusCode: 200,
505 | expectResult: true,
506 | wantContentKey: "name",
507 | },
508 | {
509 | name: "Failure - Invalid Authorization Token",
510 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-lookup-entry-tool/invoke",
511 | requestHeader: map[string]string{"my-google-auth_token": "invalid_token"},
512 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s\"}", DataplexProject, DataplexProject, DataplexProject, datasetName))),
513 | wantStatusCode: 401,
514 | expectResult: false,
515 | wantContentKey: "name",
516 | },
517 | {
518 | name: "Failure - Without Authorization Token",
519 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-lookup-entry-tool/invoke",
520 | requestHeader: map[string]string{},
521 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s\"}", DataplexProject, DataplexProject, DataplexProject, datasetName))),
522 | wantStatusCode: 401,
523 | expectResult: false,
524 | wantContentKey: "name",
525 | },
526 | {
527 | name: "Failure - Entry Not Found or Permission Denied",
528 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-lookup-entry-tool/invoke",
529 | requestHeader: map[string]string{},
530 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s\"}", DataplexProject, DataplexProject, DataplexProject, "non-existent-dataset"))),
531 | wantStatusCode: 400,
532 | expectResult: false,
533 | },
534 | {
535 | name: "Success - Entry Found with Basic View",
536 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-lookup-entry-tool/invoke",
537 | requestHeader: map[string]string{},
538 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s/tables/%s\", \"view\": %d}", DataplexProject, DataplexProject, DataplexProject, datasetName, tableName, 1))),
539 | wantStatusCode: 200,
540 | expectResult: true,
541 | wantContentKey: "name",
542 | dontWantContentKey: "aspects",
543 | },
544 | {
545 | name: "Failure - Entry with Custom View without Aspect Types",
546 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-lookup-entry-tool/invoke",
547 | requestHeader: map[string]string{},
548 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s/tables/%s\", \"view\": %d}", DataplexProject, DataplexProject, DataplexProject, datasetName, tableName, 3))),
549 | wantStatusCode: 400,
550 | expectResult: false,
551 | },
552 | {
553 | name: "Success - Entry Found with only Schema Aspect",
554 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-lookup-entry-tool/invoke",
555 | requestHeader: map[string]string{},
556 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"name\":\"projects/%s/locations/us\", \"entry\":\"projects/%s/locations/us/entryGroups/@bigquery/entries/bigquery.googleapis.com/projects/%s/datasets/%s/tables/%s\", \"aspectTypes\":[\"projects/dataplex-types/locations/global/aspectTypes/schema\"], \"view\": %d}", DataplexProject, DataplexProject, DataplexProject, datasetName, tableName, 3))),
557 | wantStatusCode: 200,
558 | expectResult: true,
559 | wantContentKey: "aspects",
560 | aspectCheck: true,
561 | },
562 | }
563 |
564 | for _, tc := range testCases {
565 | t.Run(tc.name, func(t *testing.T) {
566 | req, err := http.NewRequest(http.MethodPost, tc.api, tc.requestBody)
567 | if err != nil {
568 | t.Fatalf("unable to create request: %s", err)
569 | }
570 | req.Header.Add("Content-type", "application/json")
571 | for k, v := range tc.requestHeader {
572 | req.Header.Add(k, v)
573 | }
574 | resp, err := http.DefaultClient.Do(req)
575 | if err != nil {
576 | t.Fatalf("unable to send request: %s", err)
577 | }
578 | defer resp.Body.Close()
579 |
580 | if resp.StatusCode != tc.wantStatusCode {
581 | bodyBytes, _ := io.ReadAll(resp.Body)
582 | t.Fatalf("Response status code got %d, want %d\nResponse body: %s", resp.StatusCode, tc.wantStatusCode, string(bodyBytes))
583 | }
584 |
585 | var result map[string]interface{}
586 | if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
587 | t.Fatalf("Error parsing response body: %v", err)
588 | }
589 |
590 | if tc.expectResult {
591 | resultStr, ok := result["result"].(string)
592 | if !ok {
593 | t.Fatalf("Expected 'result' field to be a string on success, got %T", result["result"])
594 | }
595 | if resultStr == "" || resultStr == "{}" || resultStr == "null" {
596 | t.Fatal("Expected an entry, but got empty result")
597 | }
598 |
599 | var entry map[string]interface{}
600 | if err := json.Unmarshal([]byte(resultStr), &entry); err != nil {
601 | t.Fatalf("Error unmarshalling result string into entry map: %v", err)
602 | }
603 |
604 | if _, ok := entry[tc.wantContentKey]; !ok {
605 | t.Fatalf("Expected entry to have key '%s', but it was not found in %v", tc.wantContentKey, entry)
606 | }
607 |
608 | if _, ok := entry[tc.dontWantContentKey]; ok {
609 | t.Fatalf("Expected entry to not have key '%s', but it was found in %v", tc.dontWantContentKey, entry)
610 | }
611 |
612 | if tc.aspectCheck {
613 | // Check length of aspects
614 | aspects, ok := entry["aspects"].(map[string]interface{})
615 | if !ok {
616 | t.Fatalf("Expected 'aspects' to be a map, got %T", aspects)
617 | }
618 | if len(aspects) != 1 {
619 | t.Fatalf("Expected exactly one aspect, but got %d", len(aspects))
620 | }
621 | }
622 | } else { // Handle expected error response
623 | _, ok := result["error"]
624 | if !ok {
625 | t.Fatalf("Expected 'error' field in response, got %v", result)
626 | }
627 | }
628 | })
629 | }
630 | }
631 |
632 | func runDataplexSearchAspectTypesToolInvokeTest(t *testing.T, aspectTypeId string) {
633 | idToken, err := tests.GetGoogleIdToken(tests.ClientId)
634 | if err != nil {
635 | t.Fatalf("error getting Google ID token: %s", err)
636 | }
637 |
638 | testCases := []struct {
639 | name string
640 | api string
641 | requestHeader map[string]string
642 | requestBody io.Reader
643 | wantStatusCode int
644 | expectResult bool
645 | wantContentKey string
646 | }{
647 | {
648 | name: "Success - Aspect Type Found",
649 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-search-aspect-types-tool/invoke",
650 | requestHeader: map[string]string{},
651 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"name:%s_aspectType\"}", aspectTypeId))),
652 | wantStatusCode: 200,
653 | expectResult: true,
654 | wantContentKey: "metadata_template",
655 | },
656 | {
657 | name: "Success - Aspect Type Found with Authorization",
658 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-aspect-types-tool/invoke",
659 | requestHeader: map[string]string{"my-google-auth_token": idToken},
660 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"name:%s_aspectType\"}", aspectTypeId))),
661 | wantStatusCode: 200,
662 | expectResult: true,
663 | wantContentKey: "metadata_template",
664 | },
665 | {
666 | name: "Failure - Aspect Type Not Found",
667 | api: "http://127.0.0.1:5000/api/tool/my-dataplex-search-aspect-types-tool/invoke",
668 | requestHeader: map[string]string{},
669 | requestBody: bytes.NewBuffer([]byte(`"{\"query\":\"name:_aspectType\"}"`)),
670 | wantStatusCode: 400,
671 | expectResult: false,
672 | },
673 | {
674 | name: "Failure - Invalid Authorization Token",
675 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-aspect-types-tool/invoke",
676 | requestHeader: map[string]string{"my-google-auth_token": "invalid_token"},
677 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"name:%s_aspectType\"}", aspectTypeId))),
678 | wantStatusCode: 401,
679 | expectResult: false,
680 | },
681 | {
682 | name: "Failure - No Authorization Token",
683 | api: "http://127.0.0.1:5000/api/tool/my-auth-dataplex-search-aspect-types-tool/invoke",
684 | requestHeader: map[string]string{},
685 | requestBody: bytes.NewBuffer([]byte(fmt.Sprintf("{\"query\":\"name:%s_aspectType\"}", aspectTypeId))),
686 | wantStatusCode: 401,
687 | expectResult: false,
688 | },
689 | }
690 |
691 | for _, tc := range testCases {
692 | t.Run(tc.name, func(t *testing.T) {
693 | req, err := http.NewRequest(http.MethodPost, tc.api, tc.requestBody)
694 | if err != nil {
695 | t.Fatalf("unable to create request: %s", err)
696 | }
697 | req.Header.Add("Content-type", "application/json")
698 | for k, v := range tc.requestHeader {
699 | req.Header.Add(k, v)
700 | }
701 | resp, err := http.DefaultClient.Do(req)
702 | if err != nil {
703 | t.Fatalf("unable to send request: %s", err)
704 | }
705 | defer resp.Body.Close()
706 | if resp.StatusCode != tc.wantStatusCode {
707 | t.Fatalf("response status code is not %d. It is %d", tc.wantStatusCode, resp.StatusCode)
708 | }
709 | var result map[string]interface{}
710 | if err := json.NewDecoder(resp.Body).Decode(&result); err != nil {
711 | t.Fatalf("error parsing response body: %s", err)
712 | }
713 | resultStr, ok := result["result"].(string)
714 | if !ok {
715 | if result["result"] == nil && !tc.expectResult {
716 | return
717 | }
718 | t.Fatalf("expected 'result' field to be a string, got %T", result["result"])
719 | }
720 | if !tc.expectResult && (resultStr == "" || resultStr == "[]") {
721 | return
722 | }
723 | var entries []interface{}
724 | if err := json.Unmarshal([]byte(resultStr), &entries); err != nil {
725 | t.Fatalf("error unmarshalling result string: %v", err)
726 | }
727 |
728 | if tc.expectResult {
729 | if len(entries) != 1 {
730 | t.Fatalf("expected exactly one entry, but got %d", len(entries))
731 | }
732 | entry, ok := entries[0].(map[string]interface{})
733 | if !ok {
734 | t.Fatalf("expected entry to be a map, got %T", entries[0])
735 | }
736 | if _, ok := entry[tc.wantContentKey]; !ok {
737 | t.Fatalf("expected entry to have key '%s', but it was not found in %v", tc.wantContentKey, entry)
738 | }
739 | } else {
740 | if len(entries) != 0 {
741 | t.Fatalf("expected 0 entries, but got %d", len(entries))
742 | }
743 | }
744 | })
745 | }
746 | }
747 |
```
--------------------------------------------------------------------------------
/tests/looker/looker_integration_test.go:
--------------------------------------------------------------------------------
```go
1 | // Copyright 2024 Google LLC
2 | //
3 | // Licensed under the Apache License, Version 2.0 (the "License");
4 | // you may not use this file except in compliance with the License.
5 | // You may obtain a copy of the License at
6 | //
7 | // http://www.apache.org/licenses/LICENSE-2.0
8 | //
9 | // Unless required by applicable law or agreed to in writing, software
10 | // distributed under the License is distributed on an "AS IS" BASIS,
11 | // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | // See the License for the specific language governing permissions and
13 | // limitations under the License.
14 |
15 | package looker
16 |
17 | import (
18 | "bytes"
19 | "context"
20 | "encoding/json"
21 | "fmt"
22 | "net/http"
23 | "os"
24 | "regexp"
25 | "strings"
26 | "testing"
27 | "time"
28 |
29 | "github.com/googleapis/genai-toolbox/internal/log"
30 | "github.com/googleapis/genai-toolbox/internal/testutils"
31 | "github.com/googleapis/genai-toolbox/internal/util"
32 | "github.com/googleapis/genai-toolbox/tests"
33 | )
34 |
35 | var (
36 | LookerSourceKind = "looker"
37 | LookerBaseUrl = os.Getenv("LOOKER_BASE_URL")
38 | LookerVerifySsl = os.Getenv("LOOKER_VERIFY_SSL")
39 | LookerClientId = os.Getenv("LOOKER_CLIENT_ID")
40 | LookerClientSecret = os.Getenv("LOOKER_CLIENT_SECRET")
41 | LookerProject = os.Getenv("LOOKER_PROJECT")
42 | LookerLocation = os.Getenv("LOOKER_LOCATION")
43 | )
44 |
45 | func getLookerVars(t *testing.T) map[string]any {
46 | switch "" {
47 | case LookerBaseUrl:
48 | t.Fatal("'LOOKER_BASE_URL' not set")
49 | case LookerVerifySsl:
50 | t.Fatal("'LOOKER_VERIFY_SSL' not set")
51 | case LookerClientId:
52 | t.Fatal("'LOOKER_CLIENT_ID' not set")
53 | case LookerClientSecret:
54 | t.Fatal("'LOOKER_CLIENT_SECRET' not set")
55 | case LookerProject:
56 | t.Fatal("'LOOKER_PROJECT' not set")
57 | case LookerLocation:
58 | t.Fatal("'LOOKER_LOCATION' not set")
59 | }
60 |
61 | return map[string]any{
62 | "kind": LookerSourceKind,
63 | "base_url": LookerBaseUrl,
64 | "verify_ssl": (LookerVerifySsl == "true"),
65 | "client_id": LookerClientId,
66 | "client_secret": LookerClientSecret,
67 | "project": LookerProject,
68 | "location": LookerLocation,
69 | }
70 | }
71 |
72 | func TestLooker(t *testing.T) {
73 | sourceConfig := getLookerVars(t)
74 | ctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)
75 | defer cancel()
76 |
77 | testLogger, err := log.NewStdLogger(os.Stdout, os.Stderr, "info")
78 | if err != nil {
79 | t.Fatalf("unexpected error: %s", err)
80 | }
81 | ctx = util.WithLogger(ctx, testLogger)
82 |
83 | var args []string
84 |
85 | // Write config into a file and pass it to command
86 |
87 | toolsFile := map[string]any{
88 | "sources": map[string]any{
89 | "my-instance": sourceConfig,
90 | },
91 | "tools": map[string]any{
92 | "get_models": map[string]any{
93 | "kind": "looker-get-models",
94 | "source": "my-instance",
95 | "description": "Simple tool to test end to end functionality.",
96 | },
97 | "get_explores": map[string]any{
98 | "kind": "looker-get-explores",
99 | "source": "my-instance",
100 | "description": "Simple tool to test end to end functionality.",
101 | },
102 | "get_dimensions": map[string]any{
103 | "kind": "looker-get-dimensions",
104 | "source": "my-instance",
105 | "description": "Simple tool to test end to end functionality.",
106 | },
107 | "get_measures": map[string]any{
108 | "kind": "looker-get-measures",
109 | "source": "my-instance",
110 | "description": "Simple tool to test end to end functionality.",
111 | },
112 | "get_filters": map[string]any{
113 | "kind": "looker-get-filters",
114 | "source": "my-instance",
115 | "description": "Simple tool to test end to end functionality.",
116 | },
117 | "get_parameters": map[string]any{
118 | "kind": "looker-get-parameters",
119 | "source": "my-instance",
120 | "description": "Simple tool to test end to end functionality.",
121 | },
122 | "query": map[string]any{
123 | "kind": "looker-query",
124 | "source": "my-instance",
125 | "description": "Simple tool to test end to end functionality.",
126 | },
127 | "query_sql": map[string]any{
128 | "kind": "looker-query-sql",
129 | "source": "my-instance",
130 | "description": "Simple tool to test end to end functionality.",
131 | },
132 | "query_url": map[string]any{
133 | "kind": "looker-query-url",
134 | "source": "my-instance",
135 | "description": "Simple tool to test end to end functionality.",
136 | },
137 | "get_looks": map[string]any{
138 | "kind": "looker-get-looks",
139 | "source": "my-instance",
140 | "description": "Simple tool to test end to end functionality.",
141 | },
142 | "get_dashboards": map[string]any{
143 | "kind": "looker-get-dashboards",
144 | "source": "my-instance",
145 | "description": "Simple tool to test end to end functionality.",
146 | },
147 | "conversational_analytics": map[string]any{
148 | "kind": "looker-conversational-analytics",
149 | "source": "my-instance",
150 | "description": "Simple tool to test end to end functionality.",
151 | },
152 | "health_pulse": map[string]any{
153 | "kind": "looker-health-pulse",
154 | "source": "my-instance",
155 | "description": "Checks the health of a Looker instance by running a series of checks on the system.",
156 | },
157 | "health_analyze": map[string]any{
158 | "kind": "looker-health-analyze",
159 | "source": "my-instance",
160 | "description": "Provides analysis of a Looker instance's projects, models, or explores.",
161 | },
162 | "health_vacuum": map[string]any{
163 | "kind": "looker-health-vacuum",
164 | "source": "my-instance",
165 | "description": "Vacuums unused content from a Looker instance.",
166 | },
167 | },
168 | }
169 |
170 | cmd, cleanup, err := tests.StartCmd(ctx, toolsFile, args...)
171 | if err != nil {
172 | t.Fatalf("command initialization returned an error: %s", err)
173 | }
174 | defer cleanup()
175 |
176 | waitCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
177 | defer cancel()
178 | out, err := testutils.WaitForString(waitCtx, regexp.MustCompile(`Server ready to serve`), cmd.Out)
179 | if err != nil {
180 | t.Logf("toolbox command logs: \n%s", out)
181 | t.Fatalf("toolbox didn't start successfully: %s", err)
182 | }
183 |
184 | tests.RunToolGetTestByName(t, "get_models",
185 | map[string]any{
186 | "get_models": map[string]any{
187 | "description": "Simple tool to test end to end functionality.",
188 | "authRequired": []any{},
189 | "parameters": []any{},
190 | },
191 | },
192 | )
193 | tests.RunToolGetTestByName(t, "get_explores",
194 | map[string]any{
195 | "get_explores": map[string]any{
196 | "description": "Simple tool to test end to end functionality.",
197 | "authRequired": []any{},
198 | "parameters": []any{
199 | map[string]any{
200 | "authSources": []any{},
201 | "description": "The model containing the explores.",
202 | "name": "model",
203 | "required": true,
204 | "type": "string",
205 | },
206 | },
207 | },
208 | },
209 | )
210 | tests.RunToolGetTestByName(t, "get_dimensions",
211 | map[string]any{
212 | "get_dimensions": map[string]any{
213 | "description": "Simple tool to test end to end functionality.",
214 | "authRequired": []any{},
215 | "parameters": []any{
216 | map[string]any{
217 | "authSources": []any{},
218 | "description": "The model containing the explore.",
219 | "name": "model",
220 | "required": true,
221 | "type": "string",
222 | },
223 | map[string]any{
224 | "authSources": []any{},
225 | "description": "The explore containing the fields.",
226 | "name": "explore",
227 | "required": true,
228 | "type": "string",
229 | },
230 | },
231 | },
232 | },
233 | )
234 | tests.RunToolGetTestByName(t, "get_measures",
235 | map[string]any{
236 | "get_measures": map[string]any{
237 | "description": "Simple tool to test end to end functionality.",
238 | "authRequired": []any{},
239 | "parameters": []any{
240 | map[string]any{
241 | "authSources": []any{},
242 | "description": "The model containing the explore.",
243 | "name": "model",
244 | "required": true,
245 | "type": "string",
246 | },
247 | map[string]any{
248 | "authSources": []any{},
249 | "description": "The explore containing the fields.",
250 | "name": "explore",
251 | "required": true,
252 | "type": "string",
253 | },
254 | },
255 | },
256 | },
257 | )
258 | tests.RunToolGetTestByName(t, "get_parameters",
259 | map[string]any{
260 | "get_parameters": map[string]any{
261 | "description": "Simple tool to test end to end functionality.",
262 | "authRequired": []any{},
263 | "parameters": []any{
264 | map[string]any{
265 | "authSources": []any{},
266 | "description": "The model containing the explore.",
267 | "name": "model",
268 | "required": true,
269 | "type": "string",
270 | },
271 | map[string]any{
272 | "authSources": []any{},
273 | "description": "The explore containing the fields.",
274 | "name": "explore",
275 | "required": true,
276 | "type": "string",
277 | },
278 | },
279 | },
280 | },
281 | )
282 | tests.RunToolGetTestByName(t, "get_filters",
283 | map[string]any{
284 | "get_filters": map[string]any{
285 | "description": "Simple tool to test end to end functionality.",
286 | "authRequired": []any{},
287 | "parameters": []any{
288 | map[string]any{
289 | "authSources": []any{},
290 | "description": "The model containing the explore.",
291 | "name": "model",
292 | "required": true,
293 | "type": "string",
294 | },
295 | map[string]any{
296 | "authSources": []any{},
297 | "description": "The explore containing the fields.",
298 | "name": "explore",
299 | "required": true,
300 | "type": "string",
301 | },
302 | },
303 | },
304 | },
305 | )
306 | tests.RunToolGetTestByName(t, "query",
307 | map[string]any{
308 | "query": map[string]any{
309 | "description": "Simple tool to test end to end functionality.",
310 | "authRequired": []any{},
311 | "parameters": []any{
312 | map[string]any{
313 | "authSources": []any{},
314 | "description": "The model containing the explore.",
315 | "name": "model",
316 | "required": true,
317 | "type": "string",
318 | },
319 | map[string]any{
320 | "authSources": []any{},
321 | "description": "The explore to be queried.",
322 | "name": "explore",
323 | "required": true,
324 | "type": "string",
325 | },
326 | map[string]any{
327 | "authSources": []any{},
328 | "description": "The fields to be retrieved.",
329 | "items": map[string]any{
330 | "authSources": []any{},
331 | "description": "A field to be returned in the query",
332 | "name": "field",
333 | "required": true,
334 | "type": "string",
335 | },
336 | "name": "fields",
337 | "required": true,
338 | "type": "array",
339 | },
340 | map[string]any{
341 | "additionalProperties": true,
342 | "authSources": []any{},
343 | "description": "The filters for the query",
344 | "name": "filters",
345 | "required": false,
346 | "type": "object",
347 | },
348 | map[string]any{
349 | "authSources": []any{},
350 | "description": "The query pivots (must be included in fields as well).",
351 | "items": map[string]any{
352 | "authSources": []any{},
353 | "description": "A field to be used as a pivot in the query",
354 | "name": "pivot_field",
355 | "required": false,
356 | "type": "string",
357 | },
358 | "name": "pivots",
359 | "required": false,
360 | "type": "array",
361 | },
362 | map[string]any{
363 | "authSources": []any{},
364 | "description": "The sorts like \"field.id desc 0\".",
365 | "items": map[string]any{
366 | "authSources": []any{},
367 | "description": "A field to be used as a sort in the query",
368 | "name": "sort_field",
369 | "required": false,
370 | "type": "string",
371 | },
372 | "name": "sorts",
373 | "required": false,
374 | "type": "array",
375 | },
376 | map[string]any{
377 | "authSources": []any{},
378 | "description": "The row limit.",
379 | "name": "limit",
380 | "required": false,
381 | "type": "integer",
382 | },
383 | map[string]any{
384 | "authSources": []any{},
385 | "description": "The query timezone.",
386 | "name": "tz",
387 | "required": false,
388 | "type": "string",
389 | },
390 | },
391 | },
392 | },
393 | )
394 | tests.RunToolGetTestByName(t, "query_sql",
395 | map[string]any{
396 | "query_sql": map[string]any{
397 | "description": "Simple tool to test end to end functionality.",
398 | "authRequired": []any{},
399 | "parameters": []any{
400 | map[string]any{
401 | "authSources": []any{},
402 | "description": "The model containing the explore.",
403 | "name": "model",
404 | "required": true,
405 | "type": "string",
406 | },
407 | map[string]any{
408 | "authSources": []any{},
409 | "description": "The explore to be queried.",
410 | "name": "explore",
411 | "required": true,
412 | "type": "string",
413 | },
414 | map[string]any{
415 | "authSources": []any{},
416 | "description": "The fields to be retrieved.",
417 | "items": map[string]any{
418 | "authSources": []any{},
419 | "description": "A field to be returned in the query",
420 | "name": "field",
421 | "required": true,
422 | "type": "string",
423 | },
424 | "name": "fields",
425 | "required": true,
426 | "type": "array",
427 | },
428 | map[string]any{
429 | "additionalProperties": true,
430 | "authSources": []any{},
431 | "description": "The filters for the query",
432 | "name": "filters",
433 | "required": false,
434 | "type": "object",
435 | },
436 | map[string]any{
437 | "authSources": []any{},
438 | "description": "The query pivots (must be included in fields as well).",
439 | "items": map[string]any{
440 | "authSources": []any{},
441 | "description": "A field to be used as a pivot in the query",
442 | "name": "pivot_field",
443 | "required": false,
444 | "type": "string",
445 | },
446 | "name": "pivots",
447 | "required": false,
448 | "type": "array",
449 | },
450 | map[string]any{
451 | "authSources": []any{},
452 | "description": "The sorts like \"field.id desc 0\".",
453 | "items": map[string]any{
454 | "authSources": []any{},
455 | "description": "A field to be used as a sort in the query",
456 | "name": "sort_field",
457 | "required": false,
458 | "type": "string",
459 | },
460 | "name": "sorts",
461 | "required": false,
462 | "type": "array",
463 | },
464 | map[string]any{
465 | "authSources": []any{},
466 | "description": "The row limit.",
467 | "name": "limit",
468 | "required": false,
469 | "type": "integer",
470 | },
471 | map[string]any{
472 | "authSources": []any{},
473 | "description": "The query timezone.",
474 | "name": "tz",
475 | "required": false,
476 | "type": "string",
477 | },
478 | },
479 | },
480 | },
481 | )
482 | tests.RunToolGetTestByName(t, "query_url",
483 | map[string]any{
484 | "query_url": map[string]any{
485 | "description": "Simple tool to test end to end functionality.",
486 | "authRequired": []any{},
487 | "parameters": []any{
488 | map[string]any{
489 | "authSources": []any{},
490 | "description": "The model containing the explore.",
491 | "name": "model",
492 | "required": true,
493 | "type": "string",
494 | },
495 | map[string]any{
496 | "authSources": []any{},
497 | "description": "The explore to be queried.",
498 | "name": "explore",
499 | "required": true,
500 | "type": "string",
501 | },
502 | map[string]any{
503 | "authSources": []any{},
504 | "description": "The fields to be retrieved.",
505 | "items": map[string]any{
506 | "authSources": []any{},
507 | "description": "A field to be returned in the query",
508 | "name": "field",
509 | "required": true,
510 | "type": "string",
511 | },
512 | "name": "fields",
513 | "required": true,
514 | "type": "array",
515 | },
516 | map[string]any{
517 | "additionalProperties": true,
518 | "authSources": []any{},
519 | "description": "The filters for the query",
520 | "name": "filters",
521 | "required": false,
522 | "type": "object",
523 | },
524 | map[string]any{
525 | "authSources": []any{},
526 | "description": "The query pivots (must be included in fields as well).",
527 | "items": map[string]any{
528 | "authSources": []any{},
529 | "description": "A field to be used as a pivot in the query",
530 | "name": "pivot_field",
531 | "required": false,
532 | "type": "string",
533 | },
534 | "name": "pivots",
535 | "required": false,
536 | "type": "array",
537 | },
538 | map[string]any{
539 | "authSources": []any{},
540 | "description": "The sorts like \"field.id desc 0\".",
541 | "items": map[string]any{
542 | "authSources": []any{},
543 | "description": "A field to be used as a sort in the query",
544 | "name": "sort_field",
545 | "required": false,
546 | "type": "string",
547 | },
548 | "name": "sorts",
549 | "required": false,
550 | "type": "array",
551 | },
552 | map[string]any{
553 | "authSources": []any{},
554 | "description": "The row limit.",
555 | "name": "limit",
556 | "required": false,
557 | "type": "integer",
558 | },
559 | map[string]any{
560 | "authSources": []any{},
561 | "description": "The query timezone.",
562 | "name": "tz",
563 | "required": false,
564 | "type": "string",
565 | },
566 | map[string]any{
567 | "additionalProperties": true,
568 | "authSources": []any{},
569 | "description": "The visualization config for the query",
570 | "name": "vis_config",
571 | "required": false,
572 | "type": "object",
573 | },
574 | },
575 | },
576 | },
577 | )
578 | tests.RunToolGetTestByName(t, "get_looks",
579 | map[string]any{
580 | "get_looks": map[string]any{
581 | "description": "Simple tool to test end to end functionality.",
582 | "authRequired": []any{},
583 | "parameters": []any{
584 | map[string]any{
585 | "authSources": []any{},
586 | "description": "The title of the look.",
587 | "name": "title",
588 | "required": false,
589 | "type": "string",
590 | },
591 | map[string]any{
592 | "authSources": []any{},
593 | "description": "The description of the look.",
594 | "name": "desc",
595 | "required": false,
596 | "type": "string",
597 | },
598 | map[string]any{
599 | "authSources": []any{},
600 | "description": "The number of looks to fetch. Default 100",
601 | "name": "limit",
602 | "required": false,
603 | "type": "integer",
604 | },
605 | map[string]any{
606 | "authSources": []any{},
607 | "description": "The number of looks to skip before fetching. Default 0",
608 | "name": "offset",
609 | "required": false,
610 | "type": "integer",
611 | },
612 | },
613 | },
614 | },
615 | )
616 | tests.RunToolGetTestByName(t, "get_dashboards",
617 | map[string]any{
618 | "get_dashboards": map[string]any{
619 | "description": "Simple tool to test end to end functionality.",
620 | "authRequired": []any{},
621 | "parameters": []any{
622 | map[string]any{
623 | "authSources": []any{},
624 | "description": "The title of the dashboard.",
625 | "name": "title",
626 | "required": false,
627 | "type": "string",
628 | },
629 | map[string]any{
630 | "authSources": []any{},
631 | "description": "The description of the dashboard.",
632 | "name": "desc",
633 | "required": false,
634 | "type": "string",
635 | },
636 | map[string]any{
637 | "authSources": []any{},
638 | "description": "The number of dashboards to fetch. Default 100",
639 | "name": "limit",
640 | "required": false,
641 | "type": "integer",
642 | },
643 | map[string]any{
644 | "authSources": []any{},
645 | "description": "The number of dashboards to skip before fetching. Default 0",
646 | "name": "offset",
647 | "required": false,
648 | "type": "integer",
649 | },
650 | },
651 | },
652 | },
653 | )
654 |
655 | tests.RunToolGetTestByName(t, "conversational_analytics",
656 | map[string]any{
657 | "conversational_analytics": map[string]any{
658 | "description": "Simple tool to test end to end functionality.",
659 | "authRequired": []any{},
660 | "parameters": []any{
661 | map[string]any{
662 | "authSources": []any{},
663 | "description": "The user's question, potentially including conversation history and system instructions for context.",
664 | "name": "user_query_with_context",
665 | "required": true,
666 | "type": "string",
667 | },
668 | map[string]any{
669 | "authSources": []any{},
670 | "description": "An Array of at least one and up to 5 explore references like [{'model': 'MODEL_NAME', 'explore': 'EXPLORE_NAME'}]",
671 | "items": map[string]any{
672 | "additionalProperties": true,
673 | "authSources": []any{},
674 | "name": "explore_reference",
675 | "description": "An explore reference like {'model': 'MODEL_NAME', 'explore': 'EXPLORE_NAME'}",
676 | "required": true,
677 | "type": "object",
678 | },
679 | "name": "explore_references",
680 | "required": true,
681 | "type": "array",
682 | },
683 | },
684 | },
685 | },
686 | )
687 | tests.RunToolGetTestByName(t, "health_pulse",
688 | map[string]any{
689 | "health_pulse": map[string]any{
690 | "description": "Checks the health of a Looker instance by running a series of checks on the system.",
691 | "authRequired": []any{},
692 | "parameters": []any{
693 | map[string]any{
694 | "authSources": []any{},
695 | "description": "The health check to run. Can be either: `check_db_connections`, `check_dashboard_performance`,`check_dashboard_errors`,`check_explore_performance`,`check_schedule_failures`, or `check_legacy_features`",
696 | "name": "action",
697 | "required": true,
698 | "type": "string",
699 | },
700 | },
701 | },
702 | },
703 | )
704 | tests.RunToolGetTestByName(t, "health_analyze",
705 | map[string]any{
706 | "health_analyze": map[string]any{
707 | "description": "Provides analysis of a Looker instance's projects, models, or explores.",
708 | "authRequired": []any{},
709 | "parameters": []any{
710 | map[string]any{
711 | "authSources": []any{},
712 | "description": "The analysis to run. Can be 'projects', 'models', or 'explores'.",
713 | "name": "action",
714 | "required": true,
715 | "type": "string",
716 | },
717 | map[string]any{
718 | "authSources": []any{},
719 | "description": "The Looker project to analyze (optional).",
720 | "name": "project",
721 | "required": false,
722 | "type": "string",
723 | },
724 | map[string]any{
725 | "authSources": []any{},
726 | "description": "The Looker model to analyze (optional).",
727 | "name": "model",
728 | "required": false,
729 | "type": "string",
730 | },
731 | map[string]any{
732 | "authSources": []any{},
733 | "description": "The Looker explore to analyze (optional).",
734 | "name": "explore",
735 | "required": false,
736 | "type": "string",
737 | },
738 | map[string]any{
739 | "authSources": []any{},
740 | "description": "The timeframe in days to analyze.",
741 | "name": "timeframe",
742 | "required": false,
743 | "type": "integer",
744 | },
745 | map[string]any{
746 | "authSources": []any{},
747 | "description": "The minimum number of queries for a model or explore to be considered used.",
748 | "name": "min_queries",
749 | "required": false,
750 | "type": "integer",
751 | },
752 | },
753 | },
754 | },
755 | )
756 | tests.RunToolGetTestByName(t, "health_vacuum",
757 | map[string]any{
758 | "health_vacuum": map[string]any{
759 | "description": "Vacuums unused content from a Looker instance.",
760 | "authRequired": []any{},
761 | "parameters": []any{
762 | map[string]any{
763 | "authSources": []any{},
764 | "description": "The vacuum action to run. Can be 'models', or 'explores'.",
765 | "name": "action",
766 | "required": true,
767 | "type": "string",
768 | },
769 | map[string]any{
770 | "authSources": []any{},
771 | "description": "The Looker project to vacuum (optional).",
772 | "name": "project",
773 | "required": false,
774 | "type": "string",
775 | },
776 | map[string]any{
777 | "authSources": []any{},
778 | "description": "The Looker model to vacuum (optional).",
779 | "name": "model",
780 | "required": false,
781 | "type": "string",
782 | },
783 | map[string]any{
784 | "authSources": []any{},
785 | "description": "The Looker explore to vacuum (optional).",
786 | "name": "explore",
787 | "required": false,
788 | "type": "string",
789 | },
790 | map[string]any{
791 | "authSources": []any{},
792 | "description": "The timeframe in days to analyze.",
793 | "name": "timeframe",
794 | "required": false,
795 | "type": "integer",
796 | },
797 | map[string]any{
798 | "authSources": []any{},
799 | "description": "The minimum number of queries for a model or explore to be considered used.",
800 | "name": "min_queries",
801 | "required": false,
802 | "type": "integer",
803 | },
804 | },
805 | },
806 | },
807 | )
808 |
809 | wantResult := "{\"label\":\"System Activity\",\"name\":\"system__activity\",\"project_name\":\"system__activity\"}"
810 | tests.RunToolInvokeSimpleTest(t, "get_models", wantResult)
811 |
812 | wantResult = "{\"description\":\"Data about Look and dashboard usage, including frequency of views, favoriting, scheduling, embedding, and access via the API. Also includes details about individual Looks and dashboards.\",\"group_label\":\"System Activity\",\"label\":\"Content Usage\",\"name\":\"content_usage\"}"
813 | tests.RunToolInvokeParametersTest(t, "get_explores", []byte(`{"model": "system__activity"}`), wantResult)
814 |
815 | wantResult = "{\"description\":\"Number of times this content has been viewed via the Looker API\",\"label\":\"Content Usage API Count\",\"label_short\":\"API Count\",\"name\":\"content_usage.api_count\",\"type\":\"number\"}"
816 | tests.RunToolInvokeParametersTest(t, "get_dimensions", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
817 |
818 | wantResult = "{\"description\":\"The total number of views via the Looker API\",\"label\":\"Content Usage API Total\",\"label_short\":\"API Total\",\"name\":\"content_usage.api_total\",\"type\":\"sum\"}"
819 | tests.RunToolInvokeParametersTest(t, "get_measures", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
820 |
821 | wantResult = "[]"
822 | tests.RunToolInvokeParametersTest(t, "get_filters", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
823 |
824 | wantResult = "[]"
825 | tests.RunToolInvokeParametersTest(t, "get_parameters", []byte(`{"model": "system__activity", "explore": "content_usage"}`), wantResult)
826 |
827 | wantResult = "{\"look.count\":"
828 | tests.RunToolInvokeParametersTest(t, "query", []byte(`{"model": "system__activity", "explore": "look", "fields": ["look.count"]}`), wantResult)
829 |
830 | wantResult = "SELECT"
831 | tests.RunToolInvokeParametersTest(t, "query_sql", []byte(`{"model": "system__activity", "explore": "look", "fields": ["look.count"]}`), wantResult)
832 |
833 | wantResult = "system__activity"
834 | tests.RunToolInvokeParametersTest(t, "query_url", []byte(`{"model": "system__activity", "explore": "look", "fields": ["look.count"]}`), wantResult)
835 |
836 | // A system that is just being used for testing has no looks or dashboards
837 | wantResult = "null"
838 | tests.RunToolInvokeParametersTest(t, "get_looks", []byte(`{"title": "FOO", "desc": "BAR"}`), wantResult)
839 |
840 | wantResult = "null"
841 | tests.RunToolInvokeParametersTest(t, "get_dashboards", []byte(`{"title": "FOO", "desc": "BAR"}`), wantResult)
842 |
843 | runConversationalAnalytics(t, "system__activity", "content_usage")
844 |
845 | wantResult = "\"Connection\":\"thelook\""
846 | tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_db_connections"}`), wantResult)
847 |
848 | wantResult = "[]"
849 | tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_schedule_failures"}`), wantResult)
850 |
851 | wantResult = "[{\"Feature\":\"Unsupported in Looker (Google Cloud core)\"}]"
852 | tests.RunToolInvokeParametersTest(t, "health_pulse", []byte(`{"action": "check_legacy_features"}`), wantResult)
853 |
854 | wantResult = "\"Project\":\"the_look\""
855 | tests.RunToolInvokeParametersTest(t, "health_analyze", []byte(`{"action": "projects"}`), wantResult)
856 |
857 | wantResult = "\"Model\":\"the_look\""
858 | tests.RunToolInvokeParametersTest(t, "health_analyze", []byte(`{"action": "explores", "project": "the_look", "model": "the_look", "explore": "inventory_items"}`), wantResult)
859 |
860 | wantResult = "\"Model\":\"the_look\""
861 | tests.RunToolInvokeParametersTest(t, "health_vacuum", []byte(`{"action": "models"}`), wantResult)
862 | }
863 |
864 | func runConversationalAnalytics(t *testing.T, modelName, exploreName string) {
865 | exploreRefsJSON := fmt.Sprintf(`[{"model":"%s","explore":"%s"}]`, modelName, exploreName)
866 |
867 | var refs []map[string]any
868 | if err := json.Unmarshal([]byte(exploreRefsJSON), &refs); err != nil {
869 | t.Fatalf("failed to unmarshal explore refs: %v", err)
870 | }
871 |
872 | testCases := []struct {
873 | name string
874 | exploreRefs []map[string]any
875 | wantStatusCode int
876 | wantInResult string
877 | wantInError string
878 | }{
879 | {
880 | name: "invoke conversational analytics with explore",
881 | exploreRefs: refs,
882 | wantStatusCode: http.StatusOK,
883 | wantInResult: `Answer`,
884 | },
885 | }
886 |
887 | for _, tc := range testCases {
888 | t.Run(tc.name, func(t *testing.T) {
889 | requestBodyMap := map[string]any{
890 | "user_query_with_context": "What is in the explore?",
891 | "explore_references": tc.exploreRefs,
892 | }
893 | bodyBytes, err := json.Marshal(requestBodyMap)
894 | if err != nil {
895 | t.Fatalf("failed to marshal request body: %v", err)
896 | }
897 | url := "http://127.0.0.1:5000/api/tool/conversational_analytics/invoke"
898 | resp, bodyBytes := tests.RunRequest(t, http.MethodPost, url, bytes.NewBuffer(bodyBytes), nil)
899 |
900 | if resp.StatusCode != tc.wantStatusCode {
901 | t.Fatalf("unexpected status code: got %d, want %d. Body: %s", resp.StatusCode, tc.wantStatusCode, string(bodyBytes))
902 | }
903 |
904 | if tc.wantInResult != "" {
905 | var respBody map[string]interface{}
906 | if err := json.Unmarshal(bodyBytes, &respBody); err != nil {
907 | t.Fatalf("error parsing response body: %v", err)
908 | }
909 | got, ok := respBody["result"].(string)
910 | if !ok {
911 | t.Fatalf("unable to find result in response body")
912 | }
913 | if !strings.Contains(got, tc.wantInResult) {
914 | t.Errorf("unexpected result: got %q, want to contain %q", got, tc.wantInResult)
915 | }
916 | }
917 |
918 | if tc.wantInError != "" {
919 | if !strings.Contains(string(bodyBytes), tc.wantInError) {
920 | t.Errorf("unexpected error message: got %q, want to contain %q", string(bodyBytes), tc.wantInError)
921 | }
922 | }
923 | })
924 | }
925 | }
926 |
```
--------------------------------------------------------------------------------
/internal/prebuiltconfigs/tools/looker.yaml:
--------------------------------------------------------------------------------
```yaml
1 | # Copyright 2025 Google LLC
2 | #
3 | # Licensed under the Apache License, Version 2.0 (the "License");
4 | # you may not use this file except in compliance with the License.
5 | # You may obtain a copy of the License at
6 | #
7 | # http://www.apache.org/licenses/LICENSE-2.0
8 | #
9 | # Unless required by applicable law or agreed to in writing, software
10 | # distributed under the License is distributed on an "AS IS" BASIS,
11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12 | # See the License for the specific language governing permissions and
13 | # limitations under the License.
14 |
15 | sources:
16 | looker-source:
17 | kind: looker
18 | base_url: ${LOOKER_BASE_URL}
19 | client_id: ${LOOKER_CLIENT_ID:}
20 | client_secret: ${LOOKER_CLIENT_SECRET:}
21 | verify_ssl: ${LOOKER_VERIFY_SSL:true}
22 | timeout: 600s
23 | use_client_oauth: ${LOOKER_USE_CLIENT_OAUTH:false}
24 | show_hidden_models: ${LOOKER_SHOW_HIDDEN_MODELS:true}
25 | show_hidden_explores: ${LOOKER_SHOW_HIDDEN_EXPLORES:true}
26 | show_hidden_fields: ${LOOKER_SHOW_HIDDEN_FIELDS:true}
27 |
28 | tools:
29 | get_models:
30 | kind: looker-get-models
31 | source: looker-source
32 | description: |
33 | The get_models tool retrieves the list of LookML models in the Looker system.
34 |
35 | It takes no parameters.
36 |
37 | get_explores:
38 | kind: looker-get-explores
39 | source: looker-source
40 | description: |
41 | The get_explores tool retrieves the list of explores defined in a LookML model
42 | in the Looker system.
43 |
44 | It takes one parameter, the model_name looked up from get_models.
45 |
46 | get_dimensions:
47 | kind: looker-get-dimensions
48 | source: looker-source
49 | description: |
50 | The get_dimensions tool retrieves the list of dimensions defined in
51 | an explore.
52 |
53 | It takes two parameters, the model_name looked up from get_models and the
54 | explore_name looked up from get_explores.
55 |
56 | If this returns a suggestions field for a dimension, the contents of suggestions
57 | can be used as filters for this field. If this returns a suggest_explore and
58 | suggest_dimension, a query against that explore and dimension can be used to find
59 | valid filters for this field.
60 |
61 | get_measures:
62 | kind: looker-get-measures
63 | source: looker-source
64 | description: |
65 | The get_measures tool retrieves the list of measures defined in
66 | an explore.
67 |
68 | It takes two parameters, the model_name looked up from get_models and the
69 | explore_name looked up from get_explores.
70 |
71 | If this returns a suggestions field for a measure, the contents of suggestions
72 | can be used as filters for this field. If this returns a suggest_explore and
73 | suggest_dimension, a query against that explore and dimension can be used to find
74 | valid filters for this field.
75 |
76 | get_filters:
77 | kind: looker-get-filters
78 | source: looker-source
79 | description: |
80 | The get_filters tool retrieves the list of filters defined in
81 | an explore.
82 |
83 | It takes two parameters, the model_name looked up from get_models and the
84 | explore_name looked up from get_explores.
85 |
86 | get_parameters:
87 | kind: looker-get-parameters
88 | source: looker-source
89 | description: |
90 | The get_parameters tool retrieves the list of parameters defined in
91 | an explore.
92 |
93 | It takes two parameters, the model_name looked up from get_models and the
94 | explore_name looked up from get_explores.
95 |
96 | query:
97 | kind: looker-query
98 | source: looker-source
99 | description: |
100 | Query Tool
101 |
102 | This tool is used to run a query against the LookML model. The
103 | model, explore, and fields list must be specified. Pivots,
104 | filters and sorts are optional.
105 |
106 | The model can be found from the get_models tool. The explore
107 | can be found from the get_explores tool passing in the model.
108 | The fields can be found from the get_dimensions, get_measures,
109 | get_filters, and get_parameters tools, passing in the model
110 | and the explore.
111 |
112 | Provide a model_id and explore_name, then a list
113 | of fields. Optionally a list of pivots can be provided.
114 | The pivots must also be included in the fields list.
115 |
116 | Filters are provided as a map of {"field.id": "condition",
117 | "field.id2": "condition2", ...}. Do not put the field.id in
118 | quotes. Filter expressions can be found at
119 | https://cloud.google.com/looker/docs/filter-expressions. There
120 | is one mistake in that, however, Use `not null` instead of `-NULL`.
121 |
122 | Sorts can be specified like [ "field.id desc 0" ].
123 |
124 | An optional row limit can be added. If not provided the limit
125 | will default to 500. "-1" can be specified for unlimited.
126 |
127 | An optional query timezone can be added. The query_timezone to
128 | will default to that of the workstation where this MCP server
129 | is running, or Etc/UTC if that can't be determined. Not all
130 | models support custom timezones.
131 |
132 | The result of the query tool is JSON
133 |
134 | query_sql:
135 | kind: looker-query-sql
136 | source: looker-source
137 | description: |
138 | Query SQL Tool
139 |
140 | This tool is used to generate the SQL that Looker would
141 | run against the underlying database. The parameters are
142 | the same as the query tool.
143 |
144 | The result of the query sql tool is SQL text.
145 |
146 | query_url:
147 | kind: looker-query-url
148 | source: looker-source
149 | description: |
150 | Query URL Tool
151 |
152 | This tool is used to generate the URL of a query in Looker.
153 | The user can then explore the query further inside Looker.
154 | The tool also returns the query_id and slug. The parameters
155 | are the same as the query tool with an additional vis_config
156 | parameter.
157 |
158 | The vis_config is optional. If provided, it will be used to
159 | control the default visualization for the query. Here are
160 | some notes on making visualizations.
161 |
162 | ### Cartesian Charts (Area, Bar, Column, Line, Scatter)
163 |
164 | These chart types share a large number of configuration options.
165 |
166 | **General**
167 | * `type`: The type of visualization (`looker_area`, `looker_bar`, `looker_column`, `looker_line`, `looker_scatter`).
168 | * `series_types`: Override the chart type for individual series.
169 | * `show_view_names`: Display view names in labels and tooltips (`true`/`false`).
170 | * `series_labels`: Provide custom names for series.
171 |
172 | **Styling & Colors**
173 | * `colors`: An array of color values to be used for the chart series.
174 | * `series_colors`: A mapping of series names to specific color values.
175 | * `color_application`: Advanced controls for color palette application (collection, palette, reverse, etc.).
176 | * `font_size`: Font size for labels (e.g., '12px').
177 |
178 | **Legend**
179 | * `hide_legend`: Show or hide the chart legend (`true`/`false`).
180 | * `legend_position`: Placement of the legend (`'center'`, `'left'`, `'right'`).
181 |
182 | **Axes**
183 | * `swap_axes`: Swap the X and Y axes (`true`/`false`).
184 | * `x_axis_scale`: Scale of the x-axis (`'auto'`, `'ordinal'`, `'linear'`, `'time'`).
185 | * `x_axis_reversed`, `y_axis_reversed`: Reverse the direction of an axis (`true`/`false`).
186 | * `x_axis_gridlines`, `y_axis_gridlines`: Display gridlines for an axis (`true`/`false`).
187 | * `show_x_axis_label`, `show_y_axis_label`: Show or hide the axis title (`true`/`false`).
188 | * `show_x_axis_ticks`, `show_y_axis_ticks`: Show or hide axis tick marks (`true`/`false`).
189 | * `x_axis_label`, `y_axis_label`: Set a custom title for an axis.
190 | * `x_axis_datetime_label`: A format string for datetime labels on the x-axis (e.g., `'%Y-%m'`).
191 | * `x_padding_left`, `x_padding_right`: Adjust padding on the ends of the x-axis.
192 | * `x_axis_label_rotation`, `x_axis_label_rotation_bar`: Set rotation for x-axis labels.
193 | * `x_axis_zoom`, `y_axis_zoom`: Enable zooming on an axis (`true`/`false`).
194 | * `y_axes`: An array of configuration objects for multiple y-axes.
195 |
196 | **Data & Series**
197 | * `stacking`: How to stack series (`''` for none, `'normal'`, `'percent'`).
198 | * `ordering`: Order of series in a stack (`'none'`, etc.).
199 | * `limit_displayed_rows`: Enable or disable limiting the number of rows displayed (`true`/`false`).
200 | * `limit_displayed_rows_values`: Configuration for the row limit (e.g., `{ "first_last": "first", "show_hide": "show", "num_rows": 10 }`).
201 | * `discontinuous_nulls`: How to render null values in line charts (`true`/`false`).
202 | * `point_style`: Style for points on line and area charts (`'none'`, `'circle'`, `'circle_outline'`).
203 | * `series_point_styles`: Override point styles for individual series.
204 | * `interpolation`: Line interpolation style (`'linear'`, `'monotone'`, `'step'`, etc.).
205 | * `show_value_labels`: Display values on data points (`true`/`false`).
206 | * `label_value_format`: A format string for value labels.
207 | * `show_totals_labels`: Display total labels on stacked charts (`true`/`false`).
208 | * `totals_color`: Color for total labels.
209 | * `show_silhouette`: Display a "silhouette" of hidden series in stacked charts (`true`/`false`).
210 | * `hidden_series`: An array of series names to hide from the visualization.
211 |
212 | **Scatter/Bubble Specific**
213 | * `size_by_field`: The field used to determine the size of bubbles.
214 | * `color_by_field`: The field used to determine the color of bubbles.
215 | * `plot_size_by_field`: Whether to display the size-by field in the legend.
216 | * `cluster_points`: Group nearby points into clusters (`true`/`false`).
217 | * `quadrants_enabled`: Display quadrants on the chart (`true`/`false`).
218 | * `quadrant_properties`: Configuration for quadrant labels and colors.
219 | * `custom_quadrant_value_x`, `custom_quadrant_value_y`: Set quadrant boundaries as a percentage.
220 | * `custom_quadrant_point_x`, `custom_quadrant_point_y`: Set quadrant boundaries to a specific value.
221 |
222 | **Miscellaneous**
223 | * `reference_lines`: Configuration for displaying reference lines.
224 | * `trend_lines`: Configuration for displaying trend lines.
225 | * `trellis`: Configuration for creating trellis (small multiple) charts.
226 | * `crossfilterEnabled`, `crossfilters`: Configuration for cross-filtering interactions.
227 |
228 | ### Boxplot
229 |
230 | * Inherits most of the Cartesian chart options.
231 | * `type`: Must be `looker_boxplot`.
232 |
233 | ### Funnel
234 |
235 | * `type`: Must be `looker_funnel`.
236 | * `orientation`: How data is read (`'automatic'`, `'dataInRows'`, `'dataInColumns'`).
237 | * `percentType`: How percentages are calculated (`'percentOfMaxValue'`, `'percentOfPriorRow'`).
238 | * `labelPosition`, `valuePosition`, `percentPosition`: Placement of labels (`'left'`, `'right'`, `'inline'`, `'hidden'`).
239 | * `labelColor`, `labelColorEnabled`: Set a custom color for labels.
240 | * `labelOverlap`: Allow labels to overlap (`true`/`false`).
241 | * `barColors`: An array of colors for the funnel steps.
242 | * `color_application`: Advanced color palette controls.
243 | * `crossfilterEnabled`, `crossfilters`: Configuration for cross-filtering.
244 |
245 | ### Pie / Donut
246 |
247 | * Pie charts must have exactly one dimension and one numerical measure.
248 | * `type`: Must be `looker_pie`.
249 | * `value_labels`: Where to display values (`'legend'`, `'labels'`).
250 | * `label_type`: The format of data labels (`'labPer'`, `'labVal'`, `'lab'`, `'val'`, `'per'`).
251 | * `start_angle`, `end_angle`: The start and end angles of the pie chart.
252 | * `inner_radius`: The inner radius, used to create a donut chart.
253 | * `series_colors`, `series_labels`: Override colors and labels for specific slices.
254 | * `color_application`: Advanced color palette controls.
255 | * `crossfilterEnabled`, `crossfilters`: Configuration for cross-filtering.
256 | * `advanced_vis_config`: A string containing JSON for advanced Highcharts configuration.
257 |
258 | ### Waterfall
259 |
260 | * Inherits most of the Cartesian chart options.
261 | * `type`: Must be `looker_waterfall`.
262 | * `up_color`: Color for positive (increasing) values.
263 | * `down_color`: Color for negative (decreasing) values.
264 | * `total_color`: Color for the total bar.
265 |
266 | ### Word Cloud
267 |
268 | * `type`: Must be `looker_wordcloud`.
269 | * `rotation`: Enable random word rotation (`true`/`false`).
270 | * `colors`: An array of colors for the words.
271 | * `color_application`: Advanced color palette controls.
272 | * `crossfilterEnabled`, `crossfilters`: Configuration for cross-filtering.
273 |
274 | These are some sample vis_config settings.
275 |
276 | A bar chart -
277 | {{
278 | "defaults_version": 1,
279 | "label_density": 25,
280 | "legend_position": "center",
281 | "limit_displayed_rows": false,
282 | "ordering": "none",
283 | "plot_size_by_field": false,
284 | "point_style": "none",
285 | "show_null_labels": false,
286 | "show_silhouette": false,
287 | "show_totals_labels": false,
288 | "show_value_labels": false,
289 | "show_view_names": false,
290 | "show_x_axis_label": true,
291 | "show_x_axis_ticks": true,
292 | "show_y_axis_labels": true,
293 | "show_y_axis_ticks": true,
294 | "stacking": "normal",
295 | "totals_color": "#808080",
296 | "trellis": "",
297 | "type": "looker_bar",
298 | "x_axis_gridlines": false,
299 | "x_axis_reversed": false,
300 | "x_axis_scale": "auto",
301 | "x_axis_zoom": true,
302 | "y_axis_combined": true,
303 | "y_axis_gridlines": true,
304 | "y_axis_reversed": false,
305 | "y_axis_scale_mode": "linear",
306 | "y_axis_tick_density": "default",
307 | "y_axis_tick_density_custom": 5,
308 | "y_axis_zoom": true
309 | }}
310 |
311 | A column chart with an option advanced_vis_config -
312 | {{
313 | "advanced_vis_config": "{ chart: { type: 'pie', spacingBottom: 50, spacingLeft: 50, spacingRight: 50, spacingTop: 50, }, legend: { enabled: false, }, plotOptions: { pie: { dataLabels: { enabled: true, format: '\u003cb\u003e{key}\u003c/b\u003e\u003cspan style=\"font-weight: normal\"\u003e - {percentage:.2f}%\u003c/span\u003e', }, showInLegend: false, }, }, series: [], }",
314 | "colors": [
315 | "grey"
316 | ],
317 | "defaults_version": 1,
318 | "hidden_fields": [],
319 | "label_density": 25,
320 | "legend_position": "center",
321 | "limit_displayed_rows": false,
322 | "note_display": "below",
323 | "note_state": "collapsed",
324 | "note_text": "Unsold inventory only",
325 | "ordering": "none",
326 | "plot_size_by_field": false,
327 | "point_style": "none",
328 | "series_colors": {},
329 | "show_null_labels": false,
330 | "show_silhouette": false,
331 | "show_totals_labels": false,
332 | "show_value_labels": true,
333 | "show_view_names": false,
334 | "show_x_axis_label": true,
335 | "show_x_axis_ticks": true,
336 | "show_y_axis_labels": true,
337 | "show_y_axis_ticks": true,
338 | "stacking": "normal",
339 | "totals_color": "#808080",
340 | "trellis": "",
341 | "type": "looker_column",
342 | "x_axis_gridlines": false,
343 | "x_axis_reversed": false,
344 | "x_axis_scale": "auto",
345 | "x_axis_zoom": true,
346 | "y_axes": [],
347 | "y_axis_combined": true,
348 | "y_axis_gridlines": true,
349 | "y_axis_reversed": false,
350 | "y_axis_scale_mode": "linear",
351 | "y_axis_tick_density": "default",
352 | "y_axis_tick_density_custom": 5,
353 | "y_axis_zoom": true
354 | }}
355 |
356 | A line chart -
357 | {{
358 | "defaults_version": 1,
359 | "hidden_pivots": {},
360 | "hidden_series": [],
361 | "interpolation": "linear",
362 | "label_density": 25,
363 | "legend_position": "center",
364 | "limit_displayed_rows": false,
365 | "plot_size_by_field": false,
366 | "point_style": "none",
367 | "series_types": {},
368 | "show_null_points": true,
369 | "show_value_labels": false,
370 | "show_view_names": false,
371 | "show_x_axis_label": true,
372 | "show_x_axis_ticks": true,
373 | "show_y_axis_labels": true,
374 | "show_y_axis_ticks": true,
375 | "stacking": "",
376 | "trellis": "",
377 | "type": "looker_line",
378 | "x_axis_gridlines": false,
379 | "x_axis_reversed": false,
380 | "x_axis_scale": "auto",
381 | "y_axis_combined": true,
382 | "y_axis_gridlines": true,
383 | "y_axis_reversed": false,
384 | "y_axis_scale_mode": "linear",
385 | "y_axis_tick_density": "default",
386 | "y_axis_tick_density_custom": 5
387 | }}
388 |
389 | An area chart -
390 | {{
391 | "defaults_version": 1,
392 | "interpolation": "linear",
393 | "label_density": 25,
394 | "legend_position": "center",
395 | "limit_displayed_rows": false,
396 | "plot_size_by_field": false,
397 | "point_style": "none",
398 | "series_types": {},
399 | "show_null_points": true,
400 | "show_silhouette": false,
401 | "show_totals_labels": false,
402 | "show_value_labels": false,
403 | "show_view_names": false,
404 | "show_x_axis_label": true,
405 | "show_x_axis_ticks": true,
406 | "show_y_axis_labels": true,
407 | "show_y_axis_ticks": true,
408 | "stacking": "normal",
409 | "totals_color": "#808080",
410 | "trellis": "",
411 | "type": "looker_area",
412 | "x_axis_gridlines": false,
413 | "x_axis_reversed": false,
414 | "x_axis_scale": "auto",
415 | "x_axis_zoom": true,
416 | "y_axis_combined": true,
417 | "y_axis_gridlines": true,
418 | "y_axis_reversed": false,
419 | "y_axis_scale_mode": "linear",
420 | "y_axis_tick_density": "default",
421 | "y_axis_tick_density_custom": 5,
422 | "y_axis_zoom": true
423 | }}
424 |
425 | A scatter plot -
426 | {{
427 | "cluster_points": false,
428 | "custom_quadrant_point_x": 5,
429 | "custom_quadrant_point_y": 5,
430 | "custom_value_label_column": "",
431 | "custom_x_column": "",
432 | "custom_y_column": "",
433 | "defaults_version": 1,
434 | "hidden_fields": [],
435 | "hidden_pivots": {},
436 | "hidden_points_if_no": [],
437 | "hidden_series": [],
438 | "interpolation": "linear",
439 | "label_density": 25,
440 | "legend_position": "center",
441 | "limit_displayed_rows": false,
442 | "limit_displayed_rows_values": {
443 | "first_last": "first",
444 | "num_rows": 0,
445 | "show_hide": "hide"
446 | },
447 | "plot_size_by_field": false,
448 | "point_style": "circle",
449 | "quadrant_properties": {
450 | "0": {
451 | "color": "",
452 | "label": "Quadrant 1"
453 | },
454 | "1": {
455 | "color": "",
456 | "label": "Quadrant 2"
457 | },
458 | "2": {
459 | "color": "",
460 | "label": "Quadrant 3"
461 | },
462 | "3": {
463 | "color": "",
464 | "label": "Quadrant 4"
465 | }
466 | },
467 | "quadrants_enabled": false,
468 | "series_labels": {},
469 | "series_types": {},
470 | "show_null_points": false,
471 | "show_value_labels": false,
472 | "show_view_names": true,
473 | "show_x_axis_label": true,
474 | "show_x_axis_ticks": true,
475 | "show_y_axis_labels": true,
476 | "show_y_axis_ticks": true,
477 | "size_by_field": "roi",
478 | "stacking": "normal",
479 | "swap_axes": true,
480 | "trellis": "",
481 | "type": "looker_scatter",
482 | "x_axis_gridlines": false,
483 | "x_axis_reversed": false,
484 | "x_axis_scale": "auto",
485 | "x_axis_zoom": true,
486 | "y_axes": [
487 | {
488 | "label": "",
489 | "orientation": "bottom",
490 | "series": [
491 | {
492 | "axisId": "Channel_0 - average_of_roi_first",
493 | "id": "Channel_0 - average_of_roi_first",
494 | "name": "Channel_0"
495 | },
496 | {
497 | "axisId": "Channel_1 - average_of_roi_first",
498 | "id": "Channel_1 - average_of_roi_first",
499 | "name": "Channel_1"
500 | },
501 | {
502 | "axisId": "Channel_2 - average_of_roi_first",
503 | "id": "Channel_2 - average_of_roi_first",
504 | "name": "Channel_2"
505 | },
506 | {
507 | "axisId": "Channel_3 - average_of_roi_first",
508 | "id": "Channel_3 - average_of_roi_first",
509 | "name": "Channel_3"
510 | },
511 | {
512 | "axisId": "Channel_4 - average_of_roi_first",
513 | "id": "Channel_4 - average_of_roi_first",
514 | "name": "Channel_4"
515 | }
516 | ],
517 | "showLabels": true,
518 | "showValues": true,
519 | "tickDensity": "custom",
520 | "tickDensityCustom": 100,
521 | "type": "linear",
522 | "unpinAxis": false
523 | }
524 | ],
525 | "y_axis_combined": true,
526 | "y_axis_gridlines": true,
527 | "y_axis_reversed": false,
528 | "y_axis_scale_mode": "linear",
529 | "y_axis_tick_density": "default",
530 | "y_axis_tick_density_custom": 5,
531 | "y_axis_zoom": true
532 | }}
533 |
534 | A single record visualization -
535 | {{
536 | "defaults_version": 1,
537 | "show_view_names": false,
538 | "type": "looker_single_record"
539 | }}
540 |
541 | A single value visualization -
542 | {{
543 | "comparison_reverse_colors": false,
544 | "comparison_type": "value", "conditional_formatting_include_nulls": false, "conditional_formatting_include_totals": false,
545 | "custom_color": "#1A73E8",
546 | "custom_color_enabled": true,
547 | "defaults_version": 1,
548 | "enable_conditional_formatting": false,
549 | "series_types": {},
550 | "show_comparison": false,
551 | "show_comparison_label": true,
552 | "show_single_value_title": true,
553 | "single_value_title": "Total Clicks",
554 | "type": "single_value"
555 | }}
556 |
557 | A Pie chart -
558 | {{
559 | "defaults_version": 1,
560 | "label_density": 25,
561 | "label_type": "labPer",
562 | "legend_position": "center",
563 | "limit_displayed_rows": false,
564 | "ordering": "none",
565 | "plot_size_by_field": false,
566 | "point_style": "none",
567 | "series_types": {},
568 | "show_null_labels": false,
569 | "show_silhouette": false,
570 | "show_totals_labels": false,
571 | "show_value_labels": false,
572 | "show_view_names": false,
573 | "show_x_axis_label": true,
574 | "show_x_axis_ticks": true,
575 | "show_y_axis_labels": true,
576 | "show_y_axis_ticks": true,
577 | "stacking": "",
578 | "totals_color": "#808080",
579 | "trellis": "",
580 | "type": "looker_pie",
581 | "value_labels": "legend",
582 | "x_axis_gridlines": false,
583 | "x_axis_reversed": false,
584 | "x_axis_scale": "auto",
585 | "y_axis_combined": true,
586 | "y_axis_gridlines": true,
587 | "y_axis_reversed": false,
588 | "y_axis_scale_mode": "linear",
589 | "y_axis_tick_density": "default",
590 | "y_axis_tick_density_custom": 5
591 | }}
592 |
593 | The result is a JSON object with the id, slug, the url, and
594 | the long_url.
595 |
596 | get_looks:
597 | kind: looker-get-looks
598 | source: looker-source
599 | description: |
600 | get_looks Tool
601 |
602 | This tool is used to search for saved looks in a Looker instance.
603 | String search params use case-insensitive matching. String search
604 | params can contain % and '_' as SQL LIKE pattern match wildcard
605 | expressions. example="dan%" will match "danger" and "Danzig" but
606 | not "David" example="D_m%" will match "Damage" and "dump".
607 |
608 | Most search params can accept "IS NULL" and "NOT NULL" as special
609 | expressions to match or exclude (respectively) rows where the
610 | column is null.
611 |
612 | The limit and offset are used to paginate the results.
613 |
614 | The result of the get_looks tool is a list of json objects.
615 |
616 | run_look:
617 | kind: looker-run-look
618 | source: looker-source
619 | description: |
620 | run_look Tool
621 |
622 | This tool runs the query associated with a look and returns
623 | the data in a JSON structure. It accepts the look_id as the
624 | parameter.
625 |
626 | make_look:
627 | kind: looker-make-look
628 | source: looker-source
629 | description: |
630 | make_look Tool
631 |
632 | This tool creates a new look in Looker, using the query
633 | parameters and the vis_config specified.
634 |
635 | Most of the parameters are the same as the query_url
636 | tool. In addition, there is a title and a description
637 | that must be provided.
638 |
639 | The newly created look will be created in the user's
640 | personal folder in looker. The look name must be unique.
641 |
642 | The result is a json document with a link to the newly
643 | created look.
644 |
645 | get_dashboards:
646 | kind: looker-get-dashboards
647 | source: looker-source
648 | description: |
649 | get_dashboards Tool
650 |
651 | This tool is used to search for saved dashboards in a Looker instance.
652 | String search params use case-insensitive matching. String search
653 | params can contain % and '_' as SQL LIKE pattern match wildcard
654 | expressions. example="dan%" will match "danger" and "Danzig" but
655 | not "David" example="D_m%" will match "Damage" and "dump".
656 | Most search params can accept "IS NULL" and "NOT NULL" as special
657 | expressions to match or exclude (respectively) rows where the
658 | column is null.
659 |
660 | The limit and offset are used to paginate the results.
661 |
662 | The result of the get_dashboards tool is a list of json objects.
663 |
664 | make_dashboard:
665 | kind: looker-make-dashboard
666 | source: looker-source
667 | description: |
668 | make_dashboard Tool
669 |
670 | This tool creates a new dashboard in Looker. The dashboard is
671 | initially empty and the add_dashboard_element tool is used to
672 | add content to the dashboard.
673 |
674 | The newly created dashboard will be created in the user's
675 | personal folder in looker. The dashboard name must be unique.
676 |
677 | The result is a json document with a link to the newly
678 | created dashboard and the id of the dashboard. Use the id
679 | when calling add_dashboard_element.
680 |
681 | add_dashboard_element:
682 | kind: looker-add-dashboard-element
683 | source: looker-source
684 | description: |
685 | add_dashboard_element Tool
686 |
687 | This tool creates a new tile in a Looker dashboard using
688 | the query parameters and the vis_config specified.
689 |
690 | Most of the parameters are the same as the query_url
691 | tool. In addition, there is a title that may be provided.
692 | The dashboard_id must be specified. That is obtained
693 | from calling make_dashboard.
694 |
695 | This tool can be called many times for one dashboard_id
696 | and the resulting tiles will be added in order.
697 |
698 | health_pulse:
699 | kind: looker-health-pulse
700 | source: looker-source
701 | description: |
702 | health-pulse Tool
703 |
704 | This tool takes the pulse of a Looker instance by taking
705 | one of the following actions:
706 | 1. `check_db_connections`,
707 | 2. `check_dashboard_performance`,
708 | 3. `check_dashboard_errors`,
709 | 4. `check_explore_performance`,
710 | 5. `check_schedule_failures`, or
711 | 6. `check_legacy_features`
712 |
713 | health_analyze:
714 | kind: looker-health-analyze
715 | source: looker-source
716 | description: |
717 | health-analyze Tool
718 |
719 | This tool calculates the usage of projects, models and explores.
720 |
721 | It accepts 6 parameters:
722 | 1. `action`: can be "projects", "models", or "explores"
723 | 2. `project`: the project to analyze (optional)
724 | 3. `model`: the model to analyze (optional)
725 | 4. `explore`: the explore to analyze (optional)
726 | 5. `timeframe`: the lookback period in days, default is 90
727 | 6. `min_queries`: the minimum number of queries to consider a resource as active, default is 1
728 |
729 | health_vacuum:
730 | kind: looker-health-vacuum
731 | source: looker-source
732 | description: |
733 | health-vacuum Tool
734 |
735 | This tool suggests models or explores that can removed
736 | because they are unused.
737 |
738 | It accepts 6 parameters:
739 | 1. `action`: can be "models" or "explores"
740 | 2. `project`: the project to vacuum (optional)
741 | 3. `model`: the model to vacuum (optional)
742 | 4. `explore`: the explore to vacuum (optional)
743 | 5. `timeframe`: the lookback period in days, default is 90
744 | 6. `min_queries`: the minimum number of queries to consider a resource as active, default is 1
745 |
746 | The result is a list of objects that are candidates for deletion.
747 |
748 | toolsets:
749 | looker_tools:
750 | - get_models
751 | - get_explores
752 | - get_dimensions
753 | - get_measures
754 | - get_filters
755 | - get_parameters
756 | - query
757 | - query_sql
758 | - query_url
759 | - get_looks
760 | - run_look
761 | - make_look
762 | - get_dashboards
763 | - make_dashboard
764 | - add_dashboard_element
765 | - health_pulse
766 | - health_analyze
767 | - health_vacuum
```