Macro for custom schema names doesn't apply in a dbt package

Question

I have an issue using custom schema names in a dbt package.

I use the Macro provided in dbt documentation.

{% macro generate_schema_name(custom_schema_name, node) -%}

    {%- set default_schema = target.schema -%}
    {%- if custom_schema_name is none -%}

        {{ default_schema }}

    {%- else -%}

        {{ default_schema }}_{{ custom_schema_name | trim }}

    {%- endif -%}

{%- endmacro %}

I put this macro in my dbt package here dbt package.

Finally I use this dbt package in another dbt project dbt project.

Here is my dbt_project.yml in my dbt project :

name: 'covid_france'
version: '0.0.1'
config-version: 2

profile: 'default'

source-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  
clean-targets:         
    - "target"
    - "dbt_modules"

My dbt_project.yml in my dbt package :

name: 'covid_france'
version: '0.0.1'
config-version: 2

profile: 'default'

source-paths: ["models"]
analysis-paths: ["analysis"]
test-paths: ["tests"]
data-paths: ["data"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]

target-path: "target"  
clean-targets:         
    - "target"
    - "dbt_modules"

models:
    covid_france:
        stg:
            materialized: table
            schema: stg
        ods:
            materialized: table
            process-airbyte-outputs:
                schema: ods
            unions:
                schema: ods
        prs:
            materialized: view

When I try to run my dbt project, it imports the dbt package but doesn't apply the macro that is supposed to remove the main schema prefix (provided in profiles.yml) from custom schema names For instance : the schema provided in my profiles.yml is "prs". I have other custom schemas named ods and stg. But when dbt run, it create prs, prs_ods and prs_stg.

The macro used to work fine when I use it directly in a dbt project (instead of putting it in a dbt package that I use in my dbt project)

Thank you in advance !

Welcome to SO. Please clarify the problem and add more details, particularly the part of the code that you think is acting out of the expected range. It's not enough to just post links. Thanks. — StarShine
I hope it's clear enough. Don't hesitate to tell me what you want me to add. Thank you. — Ray Mar

Davos Davos · Accepted Answer · 2021-07-12T04:07:53

In the docs: https://docs.getdbt.com/docs/building-a-dbt-project/building-models/using-custom-schemas#changing-the-way-dbt-generates-a-schema-name

It says:

Note: dbt ignores any custom generate_schema_name macros that are part of a package installed in your project.

So the way around that would be to create a small "shim" or thin-wrapper directly in your project that calls the macro in your package.

I think it's a bit confusing that your package project name is the same as your actual project name (both line 1 of the dbt_project.yml files), so I would name them differently for clarity.

e.g. assuming you rename the package to package_project_name, Use the macro code as you have already in your package, but in your project add another macro like

{% macro generate_schema_name(custom_schema_name, node) -%}

  {{- package_project_name.generate_schema_name(custom_schema_name, node) }}

{%- endmacro %}

Macro for custom schema names doesn't apply in a dbt package

2 Answers