ADF metadata driven copy activity , how to process multiple tables dynamically in ADF copy activity

Incrementally copy multiple tables in adf,
incrementally load from multiple tables,
ADF Transform Multiple Tables in Single Pipeline,
Copy multiple tables in Bulk with Lookup,
Copy multiple tables in bulk by using Azure Data Factory
load database tables from many different files,
Loop through multiple files in ADLS,
Loop through multiple tables from sql to ADLS using adf,
a metadata-driven approach to data repository design
metadata driven etl

[saqibkhan-ihuu]

thanks for the detailed explanation. how and where to add audit columns like load timestamp to target tables?

[TheyCalledMeT]

since most ingestion tasks typically go to a data lake, wouldn't it be intuitive to also use the data lake for the config/metadata table(s)?

[RodrigoBocanegraCruz]

Hello, is there any way to define wildcard paths in the data lake destination? for example to have a folder structure in year/month/date/*. Thank you!

[guptaashok]

What all sources are supported for this?

[ketanmehta]

@TechLake How we can add filter clause to the query?

[snad]

Hi,
From where did you get the Sales table?

[TheyCalledMeT]

19:24 really select max watermark from target table?
not select last watermark from watermark-table filtered for this table name?

[agbtttt]

Where can we give PK name for upsert logic for each table?