> For the complete documentation index, see [llms.txt](https://ask.birdie.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://ask.birdie.ai/integrations-and-data-ingestion/how-to-export-with....md). # How to export with... ## Overview While our Standard Data Export is perfect for manual, on-demand analysis, Data Forwarding is designed for teams that need to integrate Birdie's insights directly into their internal data infrastructure. With Data Forwarding, Birdie automatically delivers a full copy of your data to your preferred cloud storage or database on a daily schedule. ### Supported Destinations This section explains how to send data from Birdie to external platforms. This allows you to further enhance your experience and assume full ownership of the insights generated with Birdie. Birdie can forward data to a variety of internal systems. You can configure your export to land in: * BigQuery * [Databricks](https://ask.birdie.ai/integrations-and-data-ingestion/how-to-export-with/databricks) * [S3 / Azure / GCS](https://ask.birdie.ai/~/revisions/IRekiKy884aHbkkhPtRV/integrations-and-data-ingestion/how-to-export-with.../s3-azure-gcs) * [SFTP](https://ask.birdie.ai/integrations-and-data-ingestion/how-to-export-with/sftp) * [Snowflake](https://ask.birdie.ai/~/revisions/82QMrzVaBqzhXDJPEWZO/integrations-and-data-ingestion/how-to-export-with.../snowflake) > Permissions Requirement: To enable this feature, you must provide Birdie with credentials that have Write permissions for your selected destination. Birdie does not require delete or administrative permissions. ### Export Schedule & Behavior To ensure your internal dashboards are always up-to-date, the forwarding process is fully automated: * Frequency: Exports trigger once every 24 hours. * Customization: The specific time and timezone can be configured to align with your organization's data ingestion windows (e.g., 2:00 AM UTC). * Incremental or full export support: Birdie can support full exports for lighter workloads or incremental for clients that have heavy data needs. * File format: CSV by default, or Parquet on request. Parquet suits datasets with long free text, where quotes and separators make CSV parsing harder. ### File Structure & Schema All exported data follows the exact same schema as our manual CSV exports. For a detailed breakdown of fields and entities, please refer to the [Data Export Schema Documentation](https://ask.birdie.ai/integrations-and-data-ingestion/data-schema-definitions-for-birdie-export). You can also follow this documentation to export a sample manually. When delivering to file stores, files are organized by date to ensure data versioning and easy historical access. The structure follows this pattern: ``` {Destination_Path}/export/{yyyy-mm-dd}/{file_name}.csv ``` On a Parquet export the layout is the same and only the extension changes, e.g. `{Destination_Path}/export/{yyyy-mm-dd}/feedbacks.parquet`. Every column is delivered as a string, so cast the ones you need on your side. The files are Snappy compressed. Included Files: * `feedbacks.csv` * `areas.csv` * `opportunities.csv` * `area_opportunities.csv` * `collections.csv` * `workspace_collections.csv` * `sentences.csv` * `messages.csv` * `segmentations.csv` * `criteria.csv` * `reasons.csv` The set of files delivered is defined when the export is configured, so a given export receives a subset of the list above rather than all of it. `workspace_collections.csv`, `segmentations.csv`, `criteria.csv` and `reasons.csv` are the ones most often left out, and can be enabled on request. `criteria.csv`, `reasons.csv`, and `segmentations.csv` also depend on the data itself: they are only generated when the workspace has labels of the corresponding type within the export window; when there are none, the file is not present in the folder. #### File names when a window is split Exports are split by volume: a window that would exceed 100k feedbacks is delivered as several smaller time ranges, each one written as its own complete set of files. Those files carry a `{name}_{HHMMSS}-{HHMMSS}` suffix marking the range they cover, and each range is written to the folder of the date it ends on. ``` export/2026-08-16/feedbacks_000000-190000.csv export/2026-08-16/feedbacks_190000-220000.csv export/2026-08-16/feedbacks_220000-223000.csv export/2026-08-17/feedbacks_223000-240000.csv ``` Consecutive ranges are contiguous, so nothing falls between them: the end of one is the start of the next. Two things to expect from the layout: * **A folder is not a day of data.** The last range of a day lands in the next day's folder, a window that was not split carries no suffix and can cover more than 24 hours (a backfill, for example), and the same feedback comes back in later folders every time it is updated. * **The suffix is a label, not a partition key.** It is counted from the start of the day the range begins on, so the hour part goes past 24 when a range crosses midnight: `240000` is midnight, and a range from 14:00 to 02:00 of the next day reads `_140000-260000`. How to pick these files up is covered in "Loading Data Into Your Warehouse / Database" below. #### Column sets in the feedbacks file The `feedbacks` file has a stable block of columns plus a set that depends on the kinds of feedback present in each window, so the column count can differ between folders. Load it by column name and treat the optional columns as nullable; reading by position, or a fixed-schema `COPY`, will fail when the column set changes. The remaining files have stable headers. When delivering to data warehouses or database systems, Birdie will consolidate each file into a table which is deduplicated and ready to use. The table names will adhere to our file naming. For the column names, we will preserve the schema used by the manual export. ### Data Retention & Pipelines Birdie prioritizes data integrity and persistence. Our system follows a Write-Only philosophy: 1. No Overwrites: Each export is stored in a date-stamped folder, named after the date its range ends on, so a split window can start in one folder and finish in the next. Birdie will never delete or modify files from previous days. 2. Historical Record: This ensures you have a reliable historical archive of your Birdie data over time. ### Loading Data Into Your Warehouse / Database In case the integration used is a file based delivery, you will need to load the files into tables in your warehouse or database. How you do this will depend on if you're using incremental exports or full exports. #### Picking up the files Whichever strategy you use, you do not need to work out which export window a file came from, and the file name is not enough to tell you. Every file is a self-contained unit of work: 1. List `{Destination_Path}/export/` and pick up every object you have not loaded yet, instead of listing a single `{yyyy-mm-dd}` folder. Keep track of the keys already loaded, or of the delivery time of the last one. 2. Load them in delivery order, oldest first, so that a later version of a row is not overwritten by an earlier one. 3. Each file is loaded on its own. Both strategies below are safe to repeat, so a file that was already loaded, or a run that failed halfway, can simply be loaded again. Files are delivered as they are produced, and Birdie does not write an end-of-export marker, so a pipeline that reacts to new objects keeps working even while a split window is still being written. #### Full exports For files that follow a full export, we recommend the following "Truncate and Load" strategy for your data pipelines: 1. Target: Create a dedicated table for each entity type (e.g., a `workspace_collections` table). 2. Refresh: Before loading the new file, truncate the existing table. 3. Ingest: Load the file. Each one is a full snapshot rather than a slice of the window, so when a split window leaves several suffixed files the most recent one is enough, and loading them all in order is harmless because each load truncates first. 4. Verify: Check that every object you picked up was loaded. The folder date records when a file was delivered, not the day its data belongs to, so on its own it cannot confirm that a period is complete. As of today, for newly configured exports this flow is necessary for the following files: * `collections.csv` * `workspace_collections.csv` #### Incremental exports The following files support incremental based exports: * `feedbacks.csv` * `opportunities.csv` * `sentences.csv` * `messages.csv` * `areas.csv` * `area_opportunities.csv` * `criteria.csv` * `reasons.csv` * `segmentations.csv` These files are large, therefore they greatly benefit from the incremental export behaviour. Clients using the legacy export may receive these files as full exports. This means each daily export contains only the rows for feedbacks that were updated since the last export. For native connectors that export into databases (e.g., Snowflake) Birdie will handle the consolidation logic. However, if you choose to receive the files via a file integration (e.g., S3 or SFTP) you will have to consolidate these files into a warehouse of your choice. To maintain a consolidated dataset, once you've selected a database solution, your processing logic must: 1. Read one export file for the entity (e.g., `feedbacks.csv`, or one of the `feedbacks_*.csv` files when the window was split) 2. Delete existing rows in your consolidated table where the delete key matches any row in that file 3. Insert all rows from that file ```sql -- Example pseudo-SQL for incremental merge -- consolidated_feedbacks: Contains the up to date copy of all your feedbacks. -- new_export_feedbacks: External table referencing the export file being loaded. BEGIN TRANSACTION; DELETE FROM consolidated_feedbacks WHERE "ID" IN (SELECT "ID" FROM new_export_feedbacks); -- List the columns explicitly: the feedbacks column set varies between windows. INSERT INTO consolidated_feedbacks ("ID", "Source", ...) SELECT "ID", "Source", ... FROM new_export_feedbacks; ... -- consolidated_opportunities: Contains the up to date copy of all your opportunities. -- new_export_opportunities: External table referencing the export file being loaded. DELETE FROM consolidated_opportunities WHERE "Feedback ID" IN (SELECT "Feedback ID" FROM new_export_opportunities); INSERT INTO consolidated_opportunities SELECT * FROM new_export_opportunities; END TRANSACTION; ``` Use the following column for the delete operation on each file: | File | Delete Key Column | | ------------------------ | ------------------------------ | | `feedbacks.csv` | `ID` | | `opportunities.csv` | `Feedback ID` | | `sentences.csv` | `Feedback ID` | | `messages.csv` | `Feedback ID` | | `areas.csv` | `Feedback ID` | | `area_opportunities.csv` | `opportunity_id` AND `area_id` | | `criteria.csv` | `Feedback ID` | | `reasons.csv` | `Feedback ID` | | `segmentations.csv` | `Feedback ID` | The reason the delete must be performed by Feedback ID is Birdie will reexport all of the related entities (opportunities, messages, sentences, areas, criteria, reasons, segmentations) for any change made to the feedback. Since 1 feedback may have N of these entities, it's important to completely remove the rows for the entities to properly deal with deletes. Otherwise, if an insert was performed without a delete, you may get duplicate data or have relationships that exist in your copy but were deleted/removed within Birdie. `area_opportunities.csv` is a junction table without a `Feedback ID` column. Delete by matching both `opportunity_id` and `area_id` from the new file. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://ask.birdie.ai/integrations-and-data-ingestion/how-to-export-with....md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.