SBT Enhancements from MTA

This page cover enhancements in SBT from the Multi-Tenant Project. Since we have migrated our app running customers to MTA, we can now take advantage of all the features below.

TLDR — MTA SBT Features

Now, we are post MTA migration we can leverage two beneficial features
bundled sources (no more manual database mapping)
RLS parameters (no more manual mappings tables)

1. New Capabilities Unlocked by MTA

Now that we have migrated our customers to MTA, we can take advantage of new features that come with MTA. The configuration of each lives in you app’s YAML.

Bundled Sources

To eliminate manually defined source, we now support bundling tables in your source YAML. This must be defined for each table source that you want.

To bundle sources, you simply add a bundled flag to your source YAML file. Insight Cloud then automatically inserts the correct database mapping for every table in that source, for every org. No more manual mapping. If any sources are left unbundled, the platform will only ask for unknown sources upon install.

# In your source YAML file, add:
is_bundled: true

# Example: in the below all tables are bundled, except DATASHARE_BEVMO_SEEK_INSIGHTS_CUSTOMERS

sources:
  - name: koddi
    description: "Koddi Ad Data"
    database: gopuff
    schema: koddi
    tables:
      - name: impression
        is_bundled: true
      - name: click
        is_bundled: true
  - name: seek_insights
    description: "Go Puff Data"
    database: GOPUFF_COPY
    schema: seek_insights
    tables:
      - name: DATASHARE_BEVMO_SEEK_INSIGHTS_CUSTOMERS
      - name: DATASHARE_BEVMO_SEEK_INSIGHTS_PROD_CATALOG 
        is_bundled: true

Without bundled sources	With bundled sources
For every new org install, someone manually enters the source database name in the platform.	Source database mappings are auto-inserted for every org. Zero manual work.
No automated install	When all sources are bundled, adding app access to an org auto-installs the app.

RLS Parameters

To control what data each subscriber receives, SEEK_ENRICH.SEEK_ORG_FILTERS

With RLS parameters, you add a parameter to your model YAML configuration. In the platform, the UI will enforce is defined in either the Seek Admin app provisioning workflow or during app subscription flows from the app store.

# In your model YAML, add a variable and mark it as rls:
variables:
models:
    - name: gp_scorecard_category_summary
    config:
      variables:
        - name: DEPARTMENT
          display_name: Department
          type: array
          default: all_departments
          values_query: go_puff_departments
          is_rls: true

Without RLS parameters	With RLS parameters
D&A manually creates lookup tables for each org, each app, each filter. Repeats constantly.	D&A adds a new param to replace any customer filtering needs.
Manual tables must be named exactly right or the app breaks.	Data powering the param is sourced from the Publisher’s data

2. Known Issues

Issue	Detail	Owner
Prefect notification bug	After a Prefect deployment finishes, the platform is not notified. It thinks the job is still running and never loads the new data.	JPG — in progress
seek_cpg module errors	seek_cpg module errors Variant runs may fail with 'no module named seek_cpg' if the wrong version of the package was uploaded to S3 during build.	kevin - to triage
Variant Missed Run Implementation	Need to implement the existing missed run logic into the platform’s API BE for SBT 1.0 apps.	JPG - Ticket in Backlog